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PREFACE 


This  report  was  prepared  by  Applied  Science  Associates,  Inc. 

(ASA),  Valencia,  Pennsylvania,  as  an  Interim  Technical  Report  under  Air 
Force  Contract  No.  F33615-78-C-0019.  Mr.  George  R.  Purifoy,  Jr.  Is  the 
Principal  Investigator  and  Project  Director.  The  Air  Force  Human 
Resources  Laboraboty  (AFHRL),  Technical  Training  Division,  Lowry  Air 
Force  Base,  Colorado,  Is  the  sponsor.  Dr.  Edgar  A.  Smith  Is  the 
Project  Engineer. 

This  study  Is  one  of  a  series  of  related  studies  under  the 
Technical  Training  Division's  Project  2361,  Simulation  for  Maintenance 
Training.  Project  2361  Is  an  advanced  development  program  to  develop, 
demonstrate,  test,  and  evaluate  selected  applications  of  computer-based 
simulation  for  Air  Force  maintenance  training.  The  objective  of  this 
program  is  to  build  baseline  knowledges  about  techniques,  procedures, 
and  principles  necessary  for  broad  applications  of  simulation  In 
maintenance  training.  Simulator  training  devices  are  being  fabricated 
and  demonstrated  in  an  operational  training  environment  in  order  to 
establish  cost,  reliability,  and  training  effectiveness  information. 
These  data  will  contribute  to  a  determination  of  training  value  factors 
for  eventual  Air  Force  use.  Demonstration  of  the  tralning/cost-ef fec- 
tlveness  of  simulation  techniques,  coupled  with  analyses  of  effective 
simulation  management  tools,  will  provide  the  necessary  empirical  data 
to  develop  model  specifications,  design  user  handbooks,  and  to  prepare 
life  cycle  management  guides  for  the  effective  utilization  of  simu¬ 
lation  in  maintenance  training. 

Summarized  in  this  report  is  the  process  currently  in  use  by  the 
Air  Force  to  achieve  design  and  acquisition  of  maintenance  training 
simulators.  Responsibilities  for  all  portions  of  the  process  are 
defined,  specific  procedures  followed  by  the  Instructional  Systems 
Development  (ISD)  team  analysts  in  deriving  training  equipment  design 
requirements,  and  by  the  System  Project  Office  (SPO)  Training  Equipment 
Acquisition  Manager  in  procurement  are  detailed,  and  existing  problems 
related  to  training  equipment  acquisition  are  discussed. 


The  authors  also  wish  to  acknowledge  the  assistance  and  cooperation 
of  the  many  individuals  who  contributed  information  and  critiques  ideas. 
From  ASA:  Dr.  John  Policy,  William  Pieper,  and  Thomas  Elliott.  From 
the  Air  Force: 


Dr.  Edgar  A.  Smith 
Major  Dennis  Downing 
Mr.  Gary  Miller 
Dr.  Joe  Yasutake 
Dr.  Marty  Rockway 
Col.  Richard  Shelton 
Dr.  Gary  A.  Klein 
Mr.  Gerald  C.  Carroll 
Dr.  Richard  J.  Schlffler 
Mr.  Larry  J.  Ivey 
Mr.  Bob  Swab 
Capt.  Richard  Jeffries 
Nr,  Marty  Bainbridge 
Lt.  Col.  John  Rutledge 
Major  Ed  Danber 
Capt.  Steve  Johnson 

Mr.  Hilton  D.  Goldman 

Capt.  Gary  Patterson 

Lt.  Col.  Peter  W.  Stoughton 

MSgt.  Dennis  Kox 

SSgt.  A1  Bebn 

MSgt.  Kd  MacXay 

TSgt.  Bob  Harrington 

Mr.  Forrest  Wolferd 

Major  Ned  Gow 

Capt.  Bernard  Garfinkel 

Capt.  Bill  Oakley 

Maj.  David  G.  Hamlin 

S/MSgt.  Dale  Lord 

TSgt.  Steve  McKinney 

Major  Daryle  D.  Cook 

Capt.  Bill  Cole 

TSgt.  Doug  Farmer 

TSgt.  Sam  Fulton 


AFHRL/TT 


AFHRL/ASR 
ASD/ENETT 
ASD/ENECH 
ASD/ENECH 
ASD/SD24 
ASD/SD2A 
ASD/SD24 
ASD/F-15  SPO 
ASD/F-16  SPO 
ASD/ATC  Resident  Of 
and  EF-lllA  SPO 
ATC/XPT 
3700  TCHTW 

3306  TES/ATF 

•* 


SAMSO/MNTP 
4315  CCTS/CMC 


3901  SMES/MBT 
4200  TES 

4017  CCTS 
FTD-533 


Lowry  AFB 


WPAFB 

"  &  Castle  AFB 

ce 

Randolph  AFB 
Sheppard  AFB 
Edwards  AFB 

t* 

«» 

Norton  AFB 
Vandenberg  AFB 


Castle  AFB 


Hill  AFB 


TABLE  OK  CONTENTS 


SECTION  I. 


SECTION  H. 


SECTION  III. 


SECTION  IV. 


BIBLIOGRAPHY 


Page  No. 


INTRODUCTION . 1 

The  General  Problem . 1 

Definitions . 2 

MAINTENANCE  TRAINING  EQUIPMENT  PROCURF>tENT 

PROCESS  SUMMARY  .  4 

Phase  1.  Identification  of  Requirements  ...  5 

Phase  II.  Development  of  Specifications.  ...  7 

Phase  III.  Procurement . ‘I 

ISD  PROCEDURES  FOR  TRAINING  EQUIPMENT  DESIGN.  ...  11 
The  ISD  Training  Equipment  Design  Process.  ...  11 
A  Training  Equipment  Design  Process  Model.  ...  29 
Major  Problem  Areas.  . . .34 

THE  SPO  TRAINING  EQUIPMENT  ACQUISITION  PROCESS.  .  .  44 

Processes  and  Procedures  .....  .  .44 

A  Training  Equipment  Acquisition  Process 

Model . 58 

Major  Problem  Areas . hi 


64 


Hi 


LIST  OF  FIGURES 


Figure  Page  No. 

1  Classes  of  Training  Devices .  3 

2  Miijor  ISD  Models  and  Their  Relationships  to  the 

Procedures  Used  at  the  330t)th  T&ES . 13 

3  Sample  of  Contractor-Provided  Task  Data  Base 

(Computer  Printout) . IS 

4  Data  Sheet  D:  Maintenance  and  Operator  Task 

Analysis  Page  1......  .  19 

5  Data  Sheet  D;  Maintenance  and  Operator  Task 

Analysis  Page  2 . 20 

6  3306  T&ES  FORM  1  (TEST)  Jan  79 . 21 

7  3306  T&ES  FORM  2  (TEST)  Jan  79 . 22 


S  3306  TJ.ES  FORM  2b  (TEST)  Jan  79  Rationale  Checklist.  .  .  23 


9  Media  Decision  Table  .  24 

10  3306  r\ES  FORM  3  (TEST)  Jan  79 . 25 

11  Teaching  Methods  Selection  Grid . 27 

12  Decision  Sequence  for  Training  Equifunent 

Selection  and  Design . 31 

13  General  Summary  -  Headings  from  Task  Description 

Worksheet . 36 

14  Flow  Chart  Depicting  Engineering  Contributions 

to  System  Acquisition . 47-54 

15  Systems  Requirement  Analysis  (SRA)  Documentation 

and  the  Processes  They  Support . 57 

16  Maintenance  Training  Equipment  Acquisition 

Procedure  1/Dec  Is  Ion  Sequence  . . 60 


iv 


LIST  OP  TABLKS 


Ttible  ’’.-‘ii.'-lii'ii 

1  Organizational  Resj'onslbil Ity  lor 

Training  Kquipmont  Roqulromonts . (> 

2  Training  Kqviipmont  Proouromont 

Responsibility  Svitnnutry . Ii1 


V 


.  Lids..  lK.  I 


SECTION  I 


INTRODUCTION 


This  technical  report  Is  the  first  In  a  series  that  will  explore 
the  problems  of  maintenance  training  simulation  design  and  acquisition. 
It  Is  focused  on  the  existing  procedures  followed  by  Air  Force  person¬ 
nel  in  performing  Instructional  Systems  Development  (ISD)  analyses  to 
define  maintenance  training  equipment  requirements,  and  by  System 
Program  Office  (SPO)  Training  Equipment  Acquisition  Managers  in  accom¬ 
plishing  training  equipment  procurement.  Later  reports  In  this  series 
will  structure  appropriate  functional  specifications  for  the  acquisi¬ 
tion  of  maintenance  training  simulators,  will  present  handbooks  to 
guide  ISD  analysts  in  selecting  appropriate  types  of  maintenance 
training  equipment  and  in  designing  and  documenting  required  mainte¬ 
nance  training  simulator  characteristics  and  features,  and  to  guide  SPO 
Acquisition  Managers  in  preparing  Prime  Item  Specifications. 

In  this  report  both  the  ISD  and  SPO  procedures  are  described  as 
they  are  currently  accomplished.  Relevant  documentation  is  cited  and  a 
comprehensive  bibliography  is  appended.  For  each  of  the  two  sets  of 
procedures  a  general  decision  model  is  presented  as  a  reference,  and 
general  problem  areas  which  appear  to  be  degrading  the  ultimate 
cost-effectiveness  of  maintenance  simulators  are  discussed. 


The  General  Problem 


Required  maintenance  capabilities  for  Air  Force  Weapon  systems 
appear  to  be  increasing  as  the  sophistication  of  weapon  systems 
Increases.  At  the  same  time,  training  budgets  are  shrinking.  These 
factors,  and  the  relatively  short  post-training  careers  of  a  high 
percentage  of  Air  Force  maintenance  personnel,  make  an  increase  in  the 
cost-effectiveness  of  maintenance  training  essential.  The  use  of 
simulators,  as  a  major  approach  to  maintenance  training,  is  assuming 
growing  importance  as  one  thrust  toward  improvement.  Simulation,  long 
an  established  training  technique  for  system  operators,  has  a  number  of 
potential  benefits  when  applied  to  the  teaching  of  system  maintenance. 
These  benefits  include  reduced  cost.  Increased  training  equipment 
reliability,  instructlonally  effective  device  characteristics,  student 
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and  instructor  safety  when  practicing  operationally  hazardous  activ¬ 
ities,  and  the  capability  for  tailored  hands-on  practice  opportunities 
through  malfunction  insertion  and  the  creation  of  operationally  critical 
and  seldom  encountered  conditions.  However,  the  realization  of  these 
advantages  has,  to  date,  not  been  spectacular.  There  are  no  formalized 
procedures  for  maintenance  simulator  design.  This  has  resulted  in  high 
variability  in  the  cost-effectiveness  of  current  maintenance  simu'  'tors. 


Definitions 


Classes  of  Training  Devices:  This  report  adopted  the  training  device 
classification  developed  by  Kinkade  and  Wheaton^  in 
which  all  "arrangements  of  equipment  components,  apparatus 
or  materials  which  provide  conditions  that  lielp  trainees 
learn  a  task"  are  DEVICES.  Devices  are  then  sub-divided, 
as  indicated  in  Figure  1,  into  TRAINING  AIDS,  which  are  used 
by  instructors  to  present  subject  matter,  and  TRAINING 
EQUIPMENT,  on  which  trainees  practice  job/ task-related 
ac  t ivit les. 

Simulator:  A  trainer  which  provides  hands-on  practice  for  aspects  of 

the  operational  job  which  liave  been  selected  on  the  basis 
of  their  criticality  and  learning  difficulty,  with 
event s/ Ind lea t ions  reproduced  ti)  the  necessary  degree  of 
fidelity  generally  under  computer  control. 

Simulation:  The  reproduction,  in  a  training  setting,  of  appropriate 

subsets  (part  task/whole  task/integrated  task)  of 
performance  opportunities  which  require  the  same 
combination  of  mental  and  physical  skills,  and  the 
application  of  the  same  group  of  knowledges,  as  those 
required  on  the  job. 

Note  that  most,  if  not  all,  items  of  training  equipment 
simulate;  but  that  only  a  unique  subset  of  tliem  are 
defined  as  simulators. 


^Kinkade,  R.  G. ,  &  Wheaton,  G.  R. ,  Training  device  design.  In 
H.  P.  Van  Cott  &  R.  G.  Kinkade  (eds.).  Human  engineering  guide  to 
equipment  design.  Washington,  D.G.:  American  Institutes  for  Research, 
1972. 


SKCTION  II 


MAINTENANCE  TRAINING  EQUIPMENT  PROCUREMENT  PROCESS  SUMMARY 


This  section  Is  on  overview  of  the  process  currently  followed  in 
acquiring  maintenance  training  equipment.  It  provides  a  framework  from 
which  to  discuss  the  specifics  of  the  two  major  components  of  the 
acquisition  process:  the  Instructional  Systems  Development  (ISD)  team 
activities,  which  are  detailed  in  Section  III;  those  of  the  System 
Program  Office  (SPO)  Training  Equipment  Acquisition  Manager  which  are 
described  in  Section  IV. 

AFHRL-TR-78-28,  Description  of  the  Air  Force  Maintenance  Training 
Device  Acquisition  and  Utilization  Process,  provides  a  relatively 
detailed  and  complete  description  of  the  total  life  cycle  for  mainten¬ 
ance  training  equipment.  The  life  cycle  can  be  divided  into  five 
phases: 


Phase  I 
Phase  II 
Phase  Hi 
Phase  IV 
Phase  V 


Identification  of  Requirements, 
Development  of  Specifications. 
Procurement. 

Utilization  and  Support. 
Retirement . 


The  focus  of  the  current  study  is  on  the  first  three  of  these 
phases.  Phase  1  outlines  the  responsibilities  of  the  ISD  team.  Phases 
II  and  HI  describe  the  responsibilities  of  the  SPO  Training  Equipment 
Acquisition  Manager. 

Unfortunately,  there  is  not  a  single  and  anfform  procedure  for 
designing  and  acquiring  training  equipment  Ir.  the  Air  Force.  By  and 
large,  each  procurement,  through  various  mixes  of  organ! zat ions ,  follows 
different  procedures  and  results  in  different  Intermediate  products 
(requirements,  specifications,  etc.).  The  following  description  only 
illustrates  the  major  processes. 


Pli;i.st>  I 


liliMU  1 1  i cat  1  nn  of  Koqu I  romciU  s 


In  IMiaso  I,  rcqn  i  fi'iiuMil  s  for  ma  i  ntoiianct'  trainini;  i‘i|ii  1  |imci)l  arc 
devolopcil  by  difforont  ori;ant  zat  ions,  cl»‘pt‘iui  i  n^  I'rimarily  upon: 

I.  Till'  ai'iiu  i  s  i  t  i  on  moito  or  stains  ot  t  lii'  svsti'ni(s) 
to  bi‘  support  I'll  - 

a.  A  now  svsiom  (ciirront  I  v  boiny.  in'ocnri-ii 
tlironyh  an  Air  l-'orci'  Svstoms  (lommmanii-SI’O ) . 

b.  A  systt'in  out  't  aoipiisition  (a  cnrri'nt  systom 
for  which  Air  force  l.oyistics  fommaiui  has  t  lie 
program  manayenii'nl  responsibility). 

c.  Si'veral  systems  (reipiiriny  common  or  yene  ra  1  i  zab  1  e 
t  ra i n i ny  support ) . 

J.  The  locus  oi  use  lor  the  t  ra  i  n  i  n>’  eijuipmenl  - 

.1.  '•lobile  rr.iininy  Sets  (MTS)  tor  field 
'('raininy,  Del  ac Imu'iit  s  (f  l’D). 

b.  Resilient  Traininy  Kqulpment  (KTK)  tor 
Technical  Traininy  fenters  (I'Tf). 

c.  Traininy,  eiiuipnu'nt  for  Strateyic  Air  thimmanil  (,SAt"> 
t raininy  facilities.  (SAf  is  noted  as  a  separate 
locus  ot  use  because  it  maintains  distinctive 
procedures  includin.y  its  own  interpretation 

ol  ISD). 

Table  I  summarizes  the  oryan i zat ions  haviny  primary  responsi bi I i t v 
for  os  tabl  Ish  i  ny,  maintenance  traininy  equipment  requirements  for  each 
acquisition  mode  and  locus  of  use. 
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Table  1 


('ri',anlzat  loaal  Responsibility  for 
Training  Kqiilpmeiit  Re(|ii  I  rement  s 


LOCUS  OF 
USE 

ACQUISITION"'^ 

MODE 

MTS  FOR  MAJOR 
WEAPON  SYSTEMS 

RTE  S,  OTHER 
MTS 

SAC  TRAINING 
EQUIPMENT 

Nl*V\  SV'f*''’*'' 

dJOlhh  TXiLS'’ 

rnino  I  fC 

:iqOM  SMI  S  MIU 

Ss  stri>K  Out 

dJOthh  TILTS'^ 

Pi  line  f  1 

•inOKt  SMi  S  Miff 
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AKR  “iO- 1 1 ,  Tralnlny,,  Mana^’emeiit  and  I't  1 1  i /at  1  >'n  of  Trainin>; 
liQviees  .nul  AKR  ‘if'-R,  Instructional  Systems  Peve lopment ,  specify  tb.it 
all  training  equipment  requi  remi'iits  must  be  developeii  accoril  i  ng  to  ISH 
procedures,  aiiil  imply  tliat  such  requirements  evolve  directly  from  t  tu' 
ISh  process.  Kach  of  the  three  major  requirement  si'tting  organizations 
listed  in  Table  I  diH's  produce  r<*<)ul  remenls  fi>r  maintenanci'  trainers, 
and  vet  each  accomplishes  them  in  a  unique  way.  The  ISh  proci'ss  does 
not  I'rovide  a  standa r<l  i  zi'd  procinlure. 

For  new  systems,  requirement  setting  Is  initiated  very  earlv  in 
the  System  Acquisition  l.ife  Cycle  (SALC).  Ceneral  requirements  for 
training  equipment,  based  on  the  maintenance  and  training  conceits 
establlshe<l  fi)r  the  weapon  system  and  on  past  experience,  are 
initiated  for  the  Statement  of  t'perat ional  Need  (SCN)  long  before  the 
Sro  anil  the  IS!)  teams  are  formed.  While  these  estimates  are  essential 
for  long-range  planning  I’urposes,  they  are  not  based  on  definitive 
training  analyses.  This  report  recognizes  that  such  training 
equipment  requirement  estimates  are  necessary  from  the  initial  concept 
stage  of  weapon  system  development  lorward  through  all  ot  the  develop¬ 
mental  phases.  However,  until  the  ISli  process  is  initiated  the  actu.il 
training  requirements  which  should  be  supported  by  training  I'qulpment 
must  remain  speculative.  While  many  benefits  could  be  gained  by 
Improvements  in  methods  tor  deriving  early  estimates  of  lrainti\g  and 
training  equipment  requ I remiuit s ,  this  sludv  examines  the  acqulsitiiui 


■-The  IKlbth  TAKS  supports  test  (AKR  RC-lAl  programs  and  perlorms 
ISO  as  directed  by  lltj  Air  Training  Command,  Includltig  major 
modifications  to  wiMpoo  systems  which  are  out  of  acqui  s  1 1  i  oii . 


process  from  the  point  where  the  ISD  team  Is  established  and  training 
requirements  bused  on  formal  analyses  emerge. 


Each  of  the  three  primary  organizations  responsible  for  conducting 
training  and  training  equipment  requirement  analyses  for  new  systems 
makes  use  at  somewhat  different  data  bases,  information  sources, 
tectmlcal  resources,  and  analytical  procedures.  Each  has  different 
procedures  and  requirements  for  training  analysis  documentation  and 
review.  However,  each  organization  Identifies  and  lists  maintenance 
tasks,  and  derives  from  them  maintenance  training  and  maintenance 
training  equlimient  requirements. 

The  procedures  followed  to  determine  training  needs  for  systems 
out  of  acquisition  and  for  several  systems  are  likewise  different 
depending  on  which  of  the  two  organizations  performs  this  function. 
There  Is  no  requirement  that  either  of  these  procedures  (unlike  those 
governing  new  systems)  follow  formalized  ISD  or  other  systematic 
training  equipment  requirement  analysis,  (lenerally,  new  training 
equipment  requirements  for  these  systems  come  from  the  primary 
Technical  Training  Center  and  are  Identified  by  Instructor  personnel. 
Typically  they  are  based  upon  a  need  to  repl.u'e  or  upgrade  existing 
training  equipment  which  Is  damaged  or  obsolete,  or  they  are  based  upon 
instructors'  Intviltlon  about  the  types  of  training  equipment  which 
would  enable  them  to  be  more  effective.  Many  ('f  tliese  tvpes  of 
training  equipment  requirements  are  developed  in  coordination  wltli  a 
system  contractor  and  are  Implemented  through  a  contractor-initiated 
engineering  change  proposal  (ECF).  Such  an  KlH'  for  a  maintenance 
trainer  Is  forwarded  to  various  organizations  for  approval,  Includinj’ 
the  3901st  SMKS/MBT's  maintenance  training  section,  (the  primary 
training  evaluation  organization  for  SAC,  located  at  Vandenberg  Air 
Force  Base,  California)  where  the  recommendations  are  verified  and  the 
Implications  for  the  new  maintenance  trainer.  In  terms  of  Its  imjiact  on 
organization,  manning,  and  logistics,  are  reviewed. 


Phase  11.  Development  of  Specifications 


The  output  of  Phase  I,  In  whatever  form  it  Is  jirepared.  Is  a 
statement  of  requirements  for  maintenance  training  equipment.  Phase 
11  translates  these  requirements  Into  specifications  appropriate  for 
contractor  design  and  fabrication.  Procedures  wlUch  structure  the 
development  of  such  specifications  fi>r  maintenance  simulators  are 
currently  uncertain,  and  are  in  a  state  of  cliange  wltliln  tlie  Air 
Force.  Only  a  few  maintenance  training  simulators  have  been  procured. 
Orgaitl zat  lonal  responsibilities  for  the  procurement  process  are  only 
now  emerging.  The  Simulator  System  Program  01  I  ice  (SIMSPO),  ASD/SDJs, 


Wr  i^^ht-Pat  tt^rson  Air  Korro  B;ise,  Ol)i(),  will  assumo  rc’spnns  1  hi  1 1 1  v  lor 
any  new  maintenance  simulators  hevond  those  cnrrentlv  In  procurement. 

At  the  lldhth  Test  and  Kvaluatlon  Squadron,  Kdwards  Air  Korce 
Rase,  California,  the  ISP  team  provides  the  following,  training 
equipment  requirements  at  the  end  of  Phase  1; 

1.  VlOh  T&KS  FORM  1  (TKSTl  dan  7<l;  A  compilation  .if 

behavioral  re.pil rement s  for  each  appropriate  trainer 
and/or  ir.ilnlnp,  aid. 

d.  \  proposed  ('ourse  Chart. 

1.  A  proposed  Trainlnp,  Standard, 

s,  A  draft  functl.mal  specification  m.i.leled  after  tlie 
Prime  Item  nevelo|iment  Specification. 

Training  equipment  rc<)ul  rement  s  from  the  iirimarv  Teclmlcal 
Training  Centers  ( FTCs)  are  pr.ivlded  on  Form  nOlh,  a  requisition  form 
used  to  lustlfv  the  need  for  each  Item  of  required  trainlnyt  equipment 
In  terms  of  the  Speclaltv  Training  Standard  elements  each  will 
support,  the  associated  number  of  students,  and  the  number  .if  h.nirs  .if 
use  anticipated. 

Req.tl  rements  from  the  3M01st  SMF.S/MRT  for  SAC  systems  cmsist  .if 
F.CPs  prepared  bv  the  svstem  contractor.  Tliese  K(M’s  may  be  ba.'ked  be  a 
tralnlny,  requirements  analysis,  althoup.h  to  date  formal  ISP  pr.ice.iures 
have  not  been  used.  Tlie  FCPs  specify  desired  changes  for  the  missile 
system  launch  c.introl  c.implex  which  Impact  tiie  simulator  an.l 'or  .itiu’r 
trainers. 

After  requirement  appr.ival  hv  uslnn  command,  ATC  and  tlie  SPO,  a 
Prime  Item  Pevel.ijiment  Spec  1  f  Icat  Ion  Is  ytenerated.  Source  sel  e.'t  1  .ms  , 
when  necessary,  are  made  usually  In  c.in}unctf.m  wltli  ISP  personnel 
and/.ir  the  uslny,  c.immand  .ir  I  y  I  na  t  .irs  of  th<’  n’qu  I  rement  s .  Often,  the 
primary  weapon  svstem  contractor  will  also  liave  the  res|ionsl  bl  1  1 1  v  .if 
.'.implex  trainers.  In  these  Instances,  development  and  fabrlcatl.m  mav 
be  sub.'.uU  rac  to.l . 

Purlnp,  equipment  deslyn,  a  Preliminary  Peslyn  Review  (PPRi,  a 
final  Critical  Peslyn  Review  (CPR),  as  well  as  f.irmal  and  Informal 
interaction  between  the  contract.ir,  the  SPtf,  and  the  ultimate  users, 
shape  and  approve  the  equipment  conf  lyurat  Ion.  Fol  l.ns’l  ny  C.PR, 
c.mtractual  arranyements  are  ma.le  for  equipment  fabrlcatl.m  and, 
usual Iv,  evaluation. 


Phase  111.  Procurement 


Procurement  procedures  for  maintenance  training  equipment  are 
complex,  Intricate,  and  unstandardized.  Table  2  summarizes  various 
procurement  responsibilities  for  different  kinds  of  maintenance 
training  equipment. 


I 


Trainin;'  Equipnent  Procurement  Responsibility  Summary 


SECTION  III 


ISO  PROCEDURES  FOR  TRAINING  EQUIPMENT  DESIGN 


In  this  Section  ISD  as  It  relates  to  maintenance  training 
equipment  design  requirements  Is  discussed  under  three  major  headings: 

1.  The  process  and  procedures  currently  used  by  the  ISD  Team 
at  the  3306th  T&ES. 

2.  A  general  ISD  decision  model  leading  to  the  Identification 
and  design  of  maintenance  simulators. 

3.  Problem  areas  which  appear  to  degrade  the  effectiveness  of 
currently  produced  training  equipment  requirements. 


The  ISD  Training  Equipment  Design  Process 


AFR  50-2,  Instructional  Systems  Development,  prescribes  that  the 
ISD  process  Is  to  be  applied  to  all  training  planning.  Including  that 
done  for  new  weapons  systems.  Techniques  for  general  application  are 
described  In  AFP  50-58,  Handbook  for  Designers  of  Instructional 
Systems,  Volumes  I-V,  and  are  taught  by  ATC  principally  In  the 
Instructional  System  Designers  course  given  by  the  3700th  Technical 
Training  Wing,  Sheppard  AFB,  Texas.  In  addition,  there  have  been 
developed  Interservice  Procedures  for  ISD  as  well  as  numerous  other 
versions  of  ISD  tailored  by  specific  civilian  and  military 
organizations  to  their  Individual  needs  and  preferences. 

In  general,  all  of  these  ISD  adaptions  have  In  common  a  flow  of 
analytical  and  developmental  procedures  which  start  with  an  analysis  of 
the  activities  for  which  training  Is  required  and  proceed  to: 

1.  Determine  training  requirements. 

2.  Establish  training  objectives  and  their  sequence. 

3.  Develop  performance  measurement  techniques. 

A.  Select  appropriate  methods  and  media. 


11 


5.  Develop  instructional  materials. 

6.  Conduct  and  evaluate  the  instructional  program. 

While  the  specific  nature  of  the  tasks  themselves  determines  the 
particular  characteristics  of  the  developed  curriculum  and  of  the 
training  equipment  needed  to  provide  necessary  hands-on  practice  of  the 
relevant  tasks,  the  ISD  approach  used  is  basically  the  same  for  all 
applications  for  both  new  or  existing  systems  as  well  as  for  I-Level  or 
0-Level  Tasks. 

The  3306th  T&ES  is  currently  applying  the  Air  Force  organization 
ISD  procedures  to  the  development  of  maintenance  training  and  training 
equipment  for  new  systems.  This  organization  has  a  core  of  highly 
experienced  ISD  team  personnel  and  has  evolved  an  adaptation  of  the 
general  ISD  model  that  has  been  singularly  successsful  in  meeting  Air 
Training  Command/Air  Force  Systems  Command  (ATC/AFSC)  requirements  for 
new  system  maintenance  training.  Their  ISD  expertise  is  unique  In  the 
Air  Force  and  ttieir  process  is  well  documented.^  For  these  reasons 
this  Section  reviews  the  procedures  of  the  3306th  T&ES  as  they  relate 
to  the  prescription  of  maintenance  training  and  the  design  of 
maintenance  training  equipment. 

Figure  2  illustrates  the  general  relationships  between  the 
3306th's  ISD  process  and  tht  procedural  steps  or  phases  outlined  in 
both  AFP  50-58  and  the  Interservice  Procedures  for  Instructional 
Systems  Development.  The  arrows  in  this  figure  connect  each  step  or 
phase  of  the  two  cited  ISD  procedures  to  the  most  similar  general  step 
or  specific  procedure  of  the  14-step  3306th's  process.  Arrows  crossing 
other  arrows  indicate  activities  which  are  done  in  different  sequences 
in  the  two  processes. 

Interviews,  in  addition  to  those  held  with  the  3306th  T&ES  on  new 
systems,  were  held  with  ISD  training  development  groups  at  a  number  of 
locations  where  production  and  upgrading  of  training  for  existing 
systems  are  done.^  At  none  of  tliese  other  locations  was  any  "by  the 
book”  ISD  analysis  used  in  formulating  requirements  and/or  designs  for 
training  equipment.  Interviewees  reported  that  formal  ISD  is  generally 
done  only  for  new  systems.  Factors  influencing  the  use  of  less 
analytical  methods  for  existing  systems  include: 


^  3306th  Test  and  Evaluation  Squadron,  Mission  Handbook,  ATC, 
June  1979. 


4 


See  listing  in  PREFACE,  page  i 


Figure  2.  Major  ISD  Models  and  Their  Relationships  to  the  Procedures  L'sed  at  the  3106th  T&£,S 


1.  Instructors'  lack  of  ISO  experience  and  training,  and  biases 
against  formal  analyses. 

2.  Insufficient  time  available  for  ciinducting  formal  analyses. 

3.  Lack  of  a  comprehensive  task  data  base. 

4.  Lack  of  formal  ISl)  procedural  guidance  for 
developing  training  equipment. 


Manning  and  Training  of  ISD  Teams 

The  3306th  has  a  small  cadre  of  experienced  ISl)  analysts.  Their 
assignments  typically  are  new  systems  which  are  being  developed  in  the 
"fly  before  buy"  mode.  Their  Involvement  witli  tliese  systems  usually 
begins  on  or  about  the  time  a  prototype  is  assigned  to  Kdwards  AKB  for 
test  and  evaluation.  At  times,  the  330bth  becomes  the  location  ot  ISO 
teams  for  systems  not  yet  fully  developed  to  the  prototype  stage  and/or 
not  assigned  to  Edwards  AFB  for  testing.  In  all  Instances,  personnel 
are  assigned  to  the  ISO  team  primarily  on  the  basis  of  their  selection 
as  future  Instructors  for  tt\e  initial  operational  units;  small 
percentage  is  retained  as  squadron  cadre.  The  operating  plii  losoplyv  is 
that  it  is  more  effective  to  select  and  train  individuals  to  be  ISO 
analysts  who  are  already  experienced  instructors  and  Subject  Matter 
Specialists  (SMS)  tlian  it  is  to  select  experienced  analysts  ;»nd  attempt 
to  make  of  them  systems  specialists  and  competent  instructors.  The 
advantages  of  tills  approach  include: 

1.  Experienced  instructors  generally  make  good  analysts,  since 
tliey  bring  to  tlie  ISO  process  first-hand  experience  with 
training  and  the  knowledge  of  tlie  types  of  training 
teclinlques  which  have  worked  for  them  most  effectively  in 
the  past. 

2.  As  SMSs ,  they  can  quickly  learn  tlie  specifics  of  a  new 
system  and  can  better  assure  that  all  system-unique  training 
requirements  are  Included  in  the  developmental  process. 

3.  Experienced  Instructors  are  able  to  quickly  learn 
appropriate  ISO  procedures.  Without  previous  ISO  training 
and  experience,  analysts  do  not  come  with  entrenched  ISO 
notions  which  differ  from  those  found  effective  by  the 
3306th. 

4.  The  3306th  T4ES  has  been  able  to  tailor  ISO  training  for 
incoming  instructors  to  specific  procedures  that  have  been 
found  to  work  best.  By  imposing  strict  documentation 
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roniii  romt'ut  s  within  tho  ISO  prooeilnre ,  ttu'  sqnavlron 
achlt'Vt's  staiulariHzrtt  Ion  in  the  iU*vt'lopm»'nt al  i  oi-oss 
and  maxtmlzos  the  traoeahl  I  It  v  of  tralnlnj;  develop¬ 
ment  deolslotis, 

Inst  rnetors  are  able  ti^  tak«*  detailed  knowledjte  of 
t  tte  svstem  and  their  owm  training  materials  with 
them  when  thev  go  to  the  operational  site.  ('onseipient  I  v , 
thev  have  the  best  possible  preparation  for  tit  ting 
the  oonrse  materials  to  the  speelfie  training  sltnatU'ns 
thev  encounter  and  fv>r  making  training  ct'iirse  updates  as 
the  svstem  evolves. 

There  are  several  disadvantages  to  these  priu-ediires: 

1.  A  large  portion  of  the  trained  and  experienced  SMSs 
leave  the  sipiadron  when  thev  assume  their  duties  as  the 
initial  instrncti'r  personnel  for  a  new  operat lonal  svstem. 
Tltls  re<iutres  the  s.piadron  to  replace  and  retrain  new 
people  for  each  new  svstem. 

2.  While  the  suviadron  prv'vldes  a  tailored  training  regimen 
for  Incoming  SMSs,  actual  practice  in  bv\llding  the  skills 
necossarv  for  effective  ISh  occurs  onlv  (in  "their  svstem." 
Tltev  are,  thus,  deprived  of  much  svstematic  feedback  whicl\ 
would  enable  the  training  equlpme\U  lievel 0)>ment  dt'cisioiis 
to  be  more  effective  during  subsequent  ai'pl  icat  ions  of  the 
ISO  process.  It  should  be  nv'ted,  however,  that  some 
senior  squadron  personnel  have  had  I'Xtenslve  ISO  experiei\ce 
with  a  number  previouslv  an.tlv.i'd  svstems. 

1.  SMSs  n«'w  to  the  squa.li'i'n  tend  to  configure  their  indlvldv\al 
training  cvuirses  and  the  training  equipment  desigits  whicii 
thev  pcv'iduce  lii  wavs  that  have  bi'en  suci't'ssful  tor  them  in 
the  p<tst  .  Iluis  ,  St  at  e-v'i -t  he-art  training  and  training 
I’qvilpment  tecl\niques  are  often  ui't  emploved  because 
manv  i\'>st  ructv'rs  lack  experience  witl\  or  even  k\\owledge  v'f 
'  them. 


ISO  Procedures 

llie  VlOnth  Mission  Handbook  prevli'uslv  reterenced  describes  in 
detail  all  1  s  I’f  the  procedural  steps  ol  their  ISO  process.  Vlu'se 
procedvires  define  in  greater  detail  the  five  phases  ot  the  general 
model  shown  in  Klgur*-  In  the  tollowing  p(>rtion  I'f  this  rt'port  the 

application  of  these  pinvedures  Is  describevi  in  terms  ot  I  Ite  five 
phases  of  the  general  model. 


In  I'rai'tico,  thf  m'lieral  inoiK-l  pliasos  arn  soUioin  impU'iiuMU  nil  in 
strict  sc(ino\itlal  oritur.  i'l'tt'n  analytic  acticitios  I'vnlap.  I'liis  is 
particularly  true  in  tlic  analysis  ot'  svstem  rcipi  i  remont  s  where  aiitii- 
tions  and  mod  i  t  i  cat  ions  to  the  jteiu-ral  data  base  are  otten  determined 
after  the  tiital  ISO  process  is  well  aloni;. 

Step  1.  Analyse  System  Re<.nii  rements.  I'lie  basic  task  data  base 
for  nuiny  new  systems  Is  generallv  provided  by  the  systems  v-ont  rac  t  or . 
Systems  deslktned  prior  to  about  mill  U>77  were  documented  bv  a  I'ask  and 
Skill  Aiialysis  Report,  illustrated  bv  Kiguri-  1.  Tvpicallv,  these  data 
are  In  the  form  of  a  computer  priittout  which  essi-niiallv  provides  task 
and  step  names,  and  some  very  general  information  concerninc,  task 
conditions,  criteria  for  performance,  aiut  c  r  i  t  i  v'a  1  i  t  v .  later  systems 
are  documented  by  the  ci>ntracti'r  following  Log,istics  Sui'port  Analysis 
(LSA)  data  formats.  However,  the  comparer  printouts  ot  I  SA  d.ita  are 
very  difficult  to  read  and  to  work  witlt,  and  they  currently  c.tnnot  hi- 
selectively  formatted  by  subsystem.  As  a  result  SMSs  prefer  to  work 
from  the  harid  written  copies  of  the  data  input  sheets.  Three  data 
fi)rmats  are  most  often  tised: 

1.  U.ita  Sheet  C;  Task  Analysis  Sumnuirv. 

2.  Data  Sheet  D;  M<tintenance  and  ftperator  Task  Analysis. 

3.  Data  Sheet  K:  Suppiiri  and  Test  laiuipmerit  or  Tr. tilting 

Material  Description  and  Just i f icaton. 

Figures  4  and  5  Illustrate  the  first  and  subsequent  pages  from  Dat.i 
Sheet  D. 

Seldom  are  any  of  these  data  bases  complete.  The  SMS  tanalvstl 
must  work  with  engineering  drawings,  contractor  design  personnel,  Test 
Force  personnel,  and  all  existing  technical  data,  as  appropriate,  to 
identify  the  specific  tasks  which  m.ike  up  the  on-the-job  performance 
requirements  from  which  training  will  be  derived.  Often  subsystems  are 
in  a  state  of  evolution,  making  the  Identification  and/v'r  det.tiled 
description  of  both  0-  attd  l-Level  tasks  Impractical.  Ad<iitional  tasks 
that  are  identified,  and  additional  infiirmation  describing  t.tsk 
performance  are  recorded  on  the  3  3t)n  TtiKS  Fi'RM  1  ITFSl'l  as  il  Instated 
in  Figure  b. 

Step  II.  Define  Training  Requirements.  The  using  eommand  for  the 
new  system  should  provide  to  the  ISO  team  Information  about  the  Air 
Force  Speciality  Codes  (AFSCs)  of  those  who  will  become  the  new  system 
trainees.  Their  previous  weapon  systems  experieuee  is  also  a  needed 
Input.  This  information  Is  often  not  available  when  needed  in  the  ISlt 
process.  As  a  result,  the  analysis  must  proceed  Initially  on  assump¬ 
tions  made  by  the  SMS  until  they  are  verified  or  modified  hy  both 


AT('  anil  the  usinj;  coinm.inil.  Onet*  the  specific  experience  of  proposed 
trainees  is  known  (or  assiuned),  tralninj^  standards  and  SMS  experience 
(or  interviews  witli  individuals  experienced  in  the  appropriate  areas) 
provide  the  SMS  with  overall  impressions  of  a  profile  of  entry 
cap,.h i 1 i t ies  to  which  training  must  be  geared.  With  this  profile  in 
mind,  the  SMS  examines  each  step  of  eacli  task  and  identifies  those  in 
wh i c  h ; 

1.  Ttiere  is  a  knowledv;e  or  a  skill  new  to  the  trainee. 

1.  Practice  will  he  necessarv  to  meet  on-the-io)’  per¬ 
formance  reipii  rements. 

For  each  task  or  step  meeting;  either  or  both  of  these  criteria,  there 
is  assumed  to  be  at  least  one  trainiiu;  requirement.  Tlie  3'30r>  T&KS  FORM 
2  (TKST)  is  used  to  document  each  task  and  all  training  requirements  in 
terms  of  behaviors,  teaching  steps  (groups  of  knowledges  required  for 
the  learning  of  the  primarv  steps),  the  conditions  and  criteria  which 
delimit  appropriate  behavior,  and  an  initial  estimate  of  training  time. 
Figure  7  illustrates  the  wav  in  which  FORM  2  is  prepared.  FORM  2b, 
Rationale  Checklist,  (Figure  R)  is  used  as  a  guide  to  define  training 
requirements  and  to  structure  additional  information  about  each 
requi remen t . 

Step  III.  Pevelop  Objectives  and  Determine  Media.  In  the  process 
of  filling  out  FCiRM  2,  the  SME  must  make  ii'rfR'ti«?nts  concerning 
situations  which  require  stimuli  in  addition  to  the  Instructor  and 
Technical  Orders.  The  situations  are,  in  general,  selected  on  the 
basis  of  instrvictor  preference  and  insight,  with  the  aid  of  a  media 
analogram  (sliovcn  in  Figure  9  in  the  form  of  a  decision  table  whicii 
represents  the  same  set  of  decisions  as  does  tlie  logic  f lowchart-tvpe 
analogram)  and  with  the  aid  of  the  Rationale  Checklist  (Figure  R). 
Specific  media  within  aitv  of  the  defined  classes  are  selected  from 
Instructor  preference,  from  a  more  detailed  listing  of  media  in  AFP 
S0-S8,  Volume  IV,  and 'or  from  information  about  available  training 
resources  for  the  course  vinder  preparation. 

Once  all  media  have  been  identified  in  this  way,  tlie  330b  T&F.S 
FORM  3  (TEST)  is  used  to  compile  all  behavioral  requirements  to  be 
satisfied  bv  each  tvpe  of  media  which  cannot  be  locally  manufactured. 
The  tvpes  of  media  for  which  a  FORM  3  is  prepared  include: 

1.  Transparencies. 

2.  Slides. 

1.  Charts,  diagrams,  i  I  1  us  t  rat  iinis 'drawl  ngs  . 
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Figure  7.  3306  T6ES  FORM  2  (TEST)  Jan  79 


RATIONAL'  (Hi 


TASK  NUMBER; 
10-28-22-00-X 


This  cherklist  will  be  annotated  with  a  check  mark 
for  each  behavioral  requirement  on  the  3306  TtS 
Form  2  (TEST'  that  will  be  satisfied  by  academic 
Instruction  to  provide  rationale  for  the  training 
requirement. 


REQUIREMENT  NUMBERS 


TRAINING  REQUIREMENT _ 

1.  New  knowledge  >  > 

2.  New  skill 

3.  Practice  required 

4.  Complex  activity 

5.  Condition/Criteria 

6.  Unique  manipulative  skills 

7.  New  SE 

8.  New  special  tools 

9.  Is  technical  data  available  >  • 

a.  Instr  clear  A  easily  understood  «  . 

b.  Oper  steps  in  logical  sequence  v  i 


c.  Schematics  adequate  for 
detailed  troubleshooting 

d.  Sys  units  location  identified 


lECHNICAL  TRAINING  MATERIALS  (TTM) 

10.  Is  hands-on  practice  required 
'1.  Is  air  vehicle  practical 

12.  Is  SE  required 

13.  Is  actual  equipment  required 

14.  Will  audio  only  suffice 

15.  Will  static  visual  alone  suffice 

16.  Is  static  visual 
and  audio  sufficient 


/ 

•V 

‘ 

• 

' 

1- ' 

• 

1 

i' 

1' 

' 

LLl 


Figure'  a.  non  UKS  FORM  .'b  ITFSl")  .i.iu 
K.’it  ionalo  ("borklisl 
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5.  Videotapes. 

6.  Trainers/simulators. 

7.  Actual  equipment. 

8.  Audio. 


1 

2 

3 

4 

5 

15 

7 

8 

9 

10 

HANDSON  PRACTICE  REQUIRED 

Y 

Y 

Y 

Y 

N 

N 

N 

N 

N 

N 

0  LEVEL  TASK 

Y 

N 

N 

Y 

AIRCRAFT  IS  PRACTICAL 

ACTUAL  EQUIPMENT  IS  REQUIRED 

AUDIO  QNLY  IS  SUFFICIENT 

Y 

Y 

N 

N 

Y 

N 

N 

N 

N 

N 

STATIC  VISUAL  ALONE  IS  SUFFICIENT 

Y 

Y 

N 

N 

N 

3  D  REQUIRED 

Y 

N 

Y 

N 

STATIC  VISUAL  &  AUDIO  IS  SUFFICIENT 

N 

Y 

Y 

AIRCRAFT 

TRAINER 

ACTUAL  EQUIPMENT 

AUDIO  RECORDING 

MODEL/CUTAWAY  t  AUDIO  RECORDING 

SOUND  SLIDE 

VIDEO  TAPES,  film,  ANIMATED  PANEL 

GRAPHICS  PRINI 

MODEL/CUTAWAY 

X 

X 

X 

X 

X 

X 

X 

■  ■  n 

X 

X 

X 

Figure  Media  Pee  is  Ion  Table 


A  medium  Item  which  can  be  manvif  ictured  locally  and  will  not  require  a 
monitored  procurement,  Is  not  documented  on  the  KPRM  3.  Figure  10 
Illustrates  a  typical  FORM  3. 

Each  FORM  3  (one  for  each  medium)  must  also  Include  a  description 
of  the  medium  Itself,  Its  physical  characteristics.  Its  content,  and 
Its  function.  These  descrlptjlons  are  usually  straightforward  for  all 
media,  with  the  exception  of  major  trainers.  When  the  medium  is  a 
simulator  or  a  major  piece  of  complex  training  equipment ,  the 
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description  on  the  Ki'RM  i  is  tnrther  detined  and  amplified  hy  the 
preparation  of  a  preliminary  Functional  Specification.  In  addition, 
identified  media  are  often  recorded  oit  tlu‘  SMF  lo>;,  (optional),  with 
notations  indicating  tluise  appropriate  fi»r  local  manufacture. 

Step  l\ .  Flan,  Develop  and  Validate  Instruction.  Once  t  ra i n i ng 
requlrments  and  recommended  training  equipment  havi'  been  established, 
tl\e  metliod  of  Instruction  is  selected  which  is  best  suited  to  teaciiing 
tlie  specific  tasks.  Tasks  wl\icli  generate  training  ri-qui  rements  are 
examined  to  identify  categories  of  task-level  training  requ  i  I'emcnt  s , 
inc luding: 

1.  Facts  and  definitions. 

d.  Co\^cepts. 

1.  Principles. 

4,  Procedures. 

'i.  Mental  Skills. 

(1.  I'syciiomot or  Skills. 

7,  Attitudes. 

Once  task-level  requirements  are  identified,  a  Teaciiing  Methods 
Selection  Orid  (Figure  11),  is  used  to  select  the  most  appropriati' 
metliod  for  providing  instruction  on  each  task.  Course  conti\'l 
diH'uments  are  prepared  which  include: 

1.  Course  charts,  which  summarize  the  anticipated  course  in 

terms  of  major  items  of  training  et]uipment  required  and  the 
segments  of  training  content  to  be  included  twith 
associated  training  time  estimates  and  anv  iither 
Informat  loti  relevant  to  the  course  instructional  designl. 

d.  Course  Training  Standards,  which  list  tasks,  knowledges, 
and  proficiency  codes  in  the  preferred  teaching  virder. 

3.  Plan  of  Instruction  (POI),  which  provides  a  relatively 
detailed  description  of  the  complete  course  by  utilts  of 
instruction,  criterion  objectives,  required  support 
materials  and  guidance,  instructional  unit  duration,  and 
appropriate  Course  Training  Standard  references.  This  POI 
becomes  the  lesson  plan  structure  for  the  training  course 
and  is  typically  persona  I  i  z.ed  by  each  instructor  using 
specific  annotations  to  cue  appropriate  in-class  instructvir 
and/or  student  activity. 


I  AlC-450b  TEACHING  METHODS  SELECTION  GRID 


SPECIAL  VISUAL  AIDS  A, VO  MATERIALS  PEQVJP.ED? 


(.)iu-o  a  K(>RM  i  is  pro|vu'otl  tor  earli  modium,  a  parka.vto  of  training; 
oquipmoiU  dosorlpt  ivo  information  is  assomhlod  for  tlie  Trainin); 

Hoqui  romonts  Reoommondat  i on  Roviow  Mootin>t  (TRRRM).  Ttu-  informat  ion 
pav'ka>;o  assomhlod  Inolvuios: 

1.  nni>  I'MiS  FORM  3  fl'KSr). 

3.  Froposod  Covirso  Ohart. 

f.  Proposod  (\-nirsi'  Trainiiux  Standards. 

■4.  Addition.il  infiM'mation  as  appropriate  tii  substantiate 
recommended  desl>;n  (FORM  ds,  etc.). 

Wtu'n  the  package  involves  a  ma  (or  trainer,  such  as  a  maintenance 
simulator,  it  will  also  include  a  preliminary  Functional  Specification, 
as  previously  described.  The  TRRRM  reviews  all  training  and  trai  iin>; 
equipment  recommondat ions  from  all  of  the  SMSs  vui  the  ISO  team  and 
consolidates  recommeitdat  (ot\s  and  equipment  requirements  to  derive  an 
optimum  media  package.  When  the  package  is  approved  by  ATC,  it  is 
forwarded  to  the  Slhi  for  p  roc  virement. 

Inst  met  iona  1  materials  are  prepared  following  the  I'rescr  i  pt  iims 
vif  tile  Cvuirse  control  dv'cuments  and  utilising  criterion  objectivi's 
derived  frvmi  the  aivilvses  documented  on  :V)0o  T&KS  Ft'RM  .1  (fKSl"). 
Performance  and/or  written  tests  are  prepared  for  criterion  objectives, 
.IS  outlined  in  the  I’Ol.  Instructional  materials,  consisting  of 
Technical  Orders,  programmed  text,  study  guides,  workboi'ks,  h. indents, 
etc.,  are  developed  and  integr.ited  following  the  I’Ol  structure  to  form 
,1  cohesive  and  instruct  ion.i  1  ly  effective  present.it  ion.  Matv'rials  .ire 
v.ilid.ited  .IS  they  .ire  prepared  .ind  refined  prior  to  fin.il  cvuirse 
c undue  t . 

Step  V.  Conduct  and  Kvaliiate  Instruction.  Although  this  step  is 
listed  as  a  part  of  the  total  ISlt  process,  it  typically  is  accomplished 
in  .1  form.il  tr. lining  environment  .it  .in  oporation.il  site.  Ootirse 
conduct  is  provided  by  the  SMSs  who  served  as  tiie  ISO  team  .ind  is  ilone 
under  the  responsibility  of  the  Field  Training  Wroup,  Technical 
Tr. lining  ('enter,  or  other  management  agency.  Course  materials  .ire 
revised  and  upd.ited  .is  .ippropriate  to  improve  their  et  feet  iveness  .iiul 
to  reflect  relevant  svstem  changes. 


A  Training  Kqulpment  Dostgn  Process  Model 

The  preceding  material  has  described  the  current  Air  Force  ISH 
process  for  determining  the  need  for  and  the  characterlst Ics  of  main¬ 
tenance  training  equipment.  In  this  section  the  underlying  decision 
logic  which  should  structure  the  process  of  proceeding  from  task 
Information  to  slmvilator  design  characteristics,  and  of  Identifying  ttie 
general  classes  of  Information  which  are  required  to  support  eacti 
decision  set  Is  explored.  This  approach  ot  putting  into  sequence  major 
sets  of  decisions  provides  a  general  training  equipment  design  process 
model.  The  model  Is  a  general  one  In  tiiat  decision  sets  are  described 
at  a  level  which  generalizes  across  most  of  tlie  training  development 
situations  in  which  tlie  ISD  process  could  resvilt  in  maintenance  training 
s Imulators. 

The  purpose  of  this  design  p'ocess  model  is  to  lay  the  groundwt'rk 
for  forthcoming  hierarchical  and  assoclatiw  relat  li'iishlps  between 
information  about  tasks  and  appropriate  training  equipment  character¬ 
istics  and  training  applications  by  comparing  it  to  current Iv  used 
procedures.  The  model  Is  an  extension  of  existing  ISh  procedures  and 
Is  Intended  to  load  toward: 

I.  A  determlnat lot\  of  appropriate  ISO  procedure  modifica¬ 
tions  which  will  cost-effectively  support  simulator 
dove lopmcnt . 

J.  An  Ident I f icat ion  of  the  procedural  steps  which  can  he 
effectively  aided  bv  having  a  refereiu'e  mam\>il  (.handbook j 
for  procedur.il  guid.ince  ,»nd  d.Ua. 

3.  The  spec  I  f  lc.it  ion  of  the  ch.ir.icterist  ics  .ind  specific 
content  of  appropriate  ISh  document.it  ivui  of  m.i  i  nt en.inct' 
slmul.itor  training  requirements. 


A  Decision  Sequence 

tUven  the  procodur.il  content  .ind  tlie  intent  of  existing  ISO 
guidance  material,  the  following  general  decision  .ire.is  begin  at  tlie 
point  of  determining  what  must  be  accompl  islied  on  the  job  .ind  end  by 
prescribing  the  best  method  of  documenting  the  design  .is  follows: 

1.  Determine  the  required  Job-relevant  skills  and 
knowledges, 

2.  Specify  those  skills  and  knowledges  which  must  bo 
learned  by  tr.ilnees  (.is  contrasted  to  those  which  the 
trainee  has  already  mastered  from  previous  training 
and/or  experience!. 


Klontity  tiu'so  skills  .mJ  kiiowlt'djjcs  wliu'li  i\in  hi’ 
most  o 1 1 00 1  i VO  1 V  loariu’il,  at  loast  in  part,  tliri'iijjh 
praotioo  on  an  itoni  ot  training  oi]ti  1  pmont  . 

4.  lironp  skills  aiul  knowloiij;os  by  class  or  typo  ot 
trainln>’  oqui pmont. 

5.  Spooity  tor  oaoh  trainin>t  oiiuipinont  typo  lun.’  wo  1  1 
tho  assoc  ivitoii  skills  aiul  knowloiijtos  must  bo 
loarnoil . 

ti .  Po  ormino  tho  oi\lor  in  which  spocitic  classes  ot 

traininit  onulpmonl  slivniUl  bo  omployod  to  tacilitato 
loatniiny. 

Kstablish  liosic.it  concepts  wluct)  cimsiitiito  prelim¬ 
inary  ilosc  r  ipt  ions  ot  traininjt  oqnipmont  character¬ 
istics  that  seem  to  most  ottootivoly  support  spocitic 
learning  roqui foment  s. 

S,  Develop  a  preliminary  Plait  ot  Instruction  tPOH  wliich 
integrates  training  equipment  and  all  itther  appropriate 
media  intvt  an  ettective  training  scenario. 

9.  Revise  and  final  ice  the  equipment  design  and  detail 

all  relevant  functional  characteristics  to  be  utiliced 
in  the  training  seenari>t. 

U' .  Document  all  equipment-related  training  requirements 

and  associated  training  equipment  functional  character¬ 
istics  as  the  principal  input  to  the  SPO  acquistion 
process. 

Vigure  12  depicts  this  decision  process  and  summarices  the 
principal  informat  ion.il  inputs  necessary  to  it. 

Critical  Peat  tires 

The  model  represented  in  Figure  12  highlights  a  number  ot  critical 
activities  (decision  sets)  which  influence  the  effectiveness  of  train¬ 
ing  equipment  as  it  is  ultimately  employed  in  a  training  regimen. 

The  model  also  suggests  th.it  criticality  of  these  activities  increases 
with  training  equipment  complexity,  since  the  potential  is  increased 
fiir  inappropriate  design  to  significantly  affect  training  quality,  and 
to  have  greater  cost  implications.  Critical  requirements  for  this 
decision  model  have  been  classified  into  four  categories: 
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1.  Task  Analysis 


a.  The  complete  decision  process  represented  by  the 
model  has  as  its  foundation  skills  and  knowledges 
derived  from  on-the-job  performance  requirements. 
These  skills  and  knowledges.  Ideally,  should  be 
derived  by  those  analysts  most  knowledgeable  about 
the  job  performance  requirement  implications  of 
various  hardware  designs;  personnel  subsystem 
specialists  from  the  equipment  contractor's 
workforce  who  prepare  LSA  data  prior  to  the  ISD 
effort.  Even  with  this  type  of  data  input, 
opportunity  must  be  provided  in  the  ISD  develop¬ 
ment  schedule  for  Air  Force  SMSs  to  perform 
additional  skill  and  knowledge  derivation. 
Analogous  skills  and  knowledges  must  also  be 
available  or  derivable,  for  the  anticipated 
trainee  population. 

b.  Definitive  task  descrlpclve  data  required  to 
support  downstream  decision-making.  Categories 
which  need  to  be  added  to  the  standard  LSA  data 
base  (or  to  be  more  descriptively  documented 
within  existing  data  categories)  Include: 

(1)  Performance  standards. 

(2)  The  identification  of  tasks  requiring 
major  psychomotor  skills. 

(3)  Tasks  which  must  be  done  in  conjunction 
with  other  tasks. 

(A)  Detail  in  the  procedures  for  the  selection 
of  possible  task  alternatives  (e.g., 
procedures  to  be  followed  in  contingency 
situations  or  troubleshooting  strategies 
appropriate  to  various  symptom  patterns). 

c.  Standardized  task  descriptive  verbs  are  needed  to 
increase  the  communication  reliability  of  task  and 
step  descriptions,  and  to  serve  as  the  basis  from 
which  associative  selections  are  made  relating, 
for  example,  training  equipment  characteristics  to 
types  of  maintenance  tasks. 

d.  Criteria  are  needed  to  provide  guidance  for  the 
realistic  assessment  of  practice  requirement 
implications  related  to  various  types  of  tasks  and 
steps. 


lii  «»».■  -  - 
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e.  Criteria  and  guidance  are  necessary  to  promote 

accuracy  and  consistency  In  the  classifications  of 
skills  and  knowledges  which  form  the  basis  for 
subsequent  decisions  relating  to  the  selection  and 
design  of  training  equipment. 

Training  Development 


a.  ISD-compatible  procedures  are  needed  to  encourage 
and  structure  the  simultaneous  design  of  training 
and  trainers. 

b.  Useful  and  unambiguous  criteria  are  needed  to 
structure  the  selection  of  skills  and  knowledges 
which  are  appropriate  for  learning  on  training 
equipment ,  especially  criteria  which  pinpoint 
those  for  which  simulation  is  not  only  appropriate 
but  essential. 

c.  A  procedure  is  needed  which  guides  the  identifi¬ 
cation  of  the  scope  of  training  requirements  which 
should  be  Incorporated  within  any  particular  item  of 
training  equipment  based  upon  an  identification  of 
the  optimum  order  for  meeting  training  objectives. 

Training  Equipment  Characteristics 


a.  Bases  and  principles  are  needed  to  guide  the 

making  of  training  equipment  tradeoffs  (that  is, 
the  process  of  recognizing  that  various  subsets  of 
training  objectives  can  be  effectively  realized  by 
following  more  than  one  medium  approach).  Major 
classes  of  alternatives  Include: 

(1)  Selecting  equipment  which  permit  practice  of 
fewer  tasks  or  on  part-tasks  rather  than 
incorporating  whole  tasks  or  integrated  task 
pract ice. 

(2)  Changes  in  level  of  fidelity. 

(3)  Selection  of  a  smaller  subset  of  practice 
situations  which  are  representative 
(generallzable)  of  those  needed  across  any 
set  of  equipment-related  training 

roqui remen ts. 
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b.  Guidance  Is  needed  which  describes  types  and  I 

techniques  of  simulation  as  they  relate  tu  the  i 

need  for  various  types  of  practice  situations. 

c.  Procedures  and  guidance  are  needed  to  effectively 
relate  specific  simulator  training  objectives  and 
their  associated  skill  and  knowledge  requirements 
to  designs  of  both  the  operational  characteristics 
to  be  simulated  and  the  Instructional  features 

appropriate  to  optimum  learning.  ' 

d.  Guidance  is  needed  in  determining  the  appro¬ 
priate  computer  generated/controlled  aspects  of 
maintenance  simulation  and  the  programming 
requirements  which  will  yield  appropriate  degrees 
of  flexibility  in  simulator  employment  and  in 
determining  In-house  maintenance  and  updating  of 
training  exercises. 

4 ,  Functional  Documentat ion 

a.  ISU  team  documentation  specifications  for  ; 

describing  training  equipment  (especially  I, 

maintenance  simulators)  to  initiate  tlie  SPO 

procurement  process  are  needed.  * 


Major  Problem  Areas 


The  preceding  material  of  this  section  lias  summarized  the  ISD 
process  currently  In  use  in  the  Air  Force.  Overlayed  on  this  process 
was  a  general  decision  model  structuring  the  design  and  documentat ion 
of  maintenance  training  equipment.  Contrasting  this  general  model  wltli 
Information  about  current  Air  Force  ISO  practices  has  highlighted  six 
problems  areas: 

1.  Lack  of  procedural  documentat ion. 

2.  ISl)  not  fully  applied  in  this  area. 

3.  A  Priori  simulator  selection. 

4.  insufficient  ISO  team  training. 
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■j.  Required  luformatluu  not  available, 
b.  Incomplete  analyses. 

These  problem  areas  overlap  and  Interact.  Kor  example,  ISI) 
analysts  <ire  not  trained  in  certain  areas  simply  because  no  formal 
procedures  currently  exist  on  whicli  to  base  tralnln>;.  Kach  area  is 
discussed  here,  however,  to  assist  in  conceptual  lzin>;  solutli'iis  which 
ttave  potential  for  improving  the  efflclencv  atid  the  tuyst -I't  Ifct  i  veni'SN 
of  the  ISh  process  as  it  produces  tralnln.q  equl(iment  recommendations, 
particularly  for  maintenance  simulation. 

This  classification  of  pr<)blein  areas  assoclateil  witli  the  appli¬ 
cation  of  ISO  procedures  appears,  at  first  blush,  to  be  a  heavy 
Indictment.  However,  In  revlewlnf*  the  specific  problems  with  each  of 
these  are.is,  it  is  Important  to  maintain  a  realistic  perspective.  The 
ISO  concept  is  relatively  new,  uniquely  demanding,  and  not  widely 
applied.  Kven  so.  Its  users,  particularly  the  TU'bth  TiKS,  have 
amassed  .m  Impressive  record  of  effective  irainlitg  development  and 
Implementation,  This  cl.issi f Icat Ion  of  existing  problems  needs  to  be 
taken  for  wliat  It  is,  an  attempt  to  identify  ways  in  wtilclt  an  already 
successful  process  can  be  furtlter  improved  In  the  cost-effectiveness  ot 
Its  products. 


Lack  of  Procedural  Documentation 


An  extensive  review  has  been  made  of  the  Information  requirements 
specified  for  ISO  analyses  in  AKP  S0-S8,  VoUimo  11,  Task  Analysis, 
dated  IS  .Uilv  1078.  The  desc  r  i  [>t  ions  in  tliat  volume  constitute  the 
prlm.iry  procedural  resource  for  ISO  in  the  Air  force.  The  types, 
categories,  and  overall  nature  of  the  Information  requirements 
specified  in  AFP  SO-SS  for  the  ISO  task  analysis  are  both  comprehensive 
In  terms  of  the  description  It  provides  of  tasks/ac 1 1 v i t  les  and 
sufficient  to  provide  the  structure  for  all  appropriate  training  and 
training  equipmeitt  development  decisions.  IX’tails  of  the  inlormoition 
requirements  and  the  task  analytic  procedures  will  not  be  repeated  in 
this  report.  The  headings  from  the  task  description  worksheet  usied  to 
collate  and  summarize  data  for  the  task  analysis  are  presented  In 
Figure  If  as  a  general  svimmary. 

Three  ma  lor  shortcomings  characterize  ISO  documentation  currently 
Itt  use,  including  AFP  ‘)l)-‘S8  and  all  other  ISO  reterences  previously 
c i t  ed  : 

1.  They  describe  what  Is  often  an  idealistic  data 
availability  situation.  Much  of  the  time  the 
depth,  accuracy,  ainl  reliability  of  task  data 
available  to  the  ISO  analyst  does  not  permit 
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clean  abstraction  into  the  specified  categories 
of  the  analysis  procedure.  Few  suggestions  are 
provided  to  assist  the  ISD  team  member  in  these 
situations. 

2.  The  ISD  guides  and  handbooks  available  are  "principle" 
oriented  and  provide  minimum  guidance  on  the  procedural 
or  mechanistic  application  of  these  principles  in  making 
training  and  training  device  design  decisions. 

3.  The  nature  of  the  decision-making  required  for  ISD 
necessitates  the  learning,  retention,  and  Integration 
of  a  large  number  of  complex  concepts/constructs  and 
associated  knowledges.  The  procedures,  In  total, 
constitute  a  set  of  skill  requirements  which  necessitates 
extensive  practice  for  mastery.  ISD,  In  Its  intended 
sense,  cannot  be  conducted  by  Individuals,  no  matter 

how  well  motivated  and  operationally  knowledgeable, 
who  have  not  had  an  opportunity  for  extensive  practice 
and  insightful  feedback.  The  ISD  process  Is  neither 
mystical  nor  extremely  difficult.  However,  It  requires 
an  ability  to  conceptually  manipulate  and  dissect 
behavioral  Information.  This  Is  foreign  to  many  teaching/ 
training  situations.  It  demands  a  degree  of  metlcu- 
lousness  and  exhaustiveness  with  the  minutiae  of  tasks 
and  activities  in  order  to  make  the  same  kinds  of  training 
decisions  that  are  typically  made  with  far  less  rigor. 


ISD  Not  Fully  Applied 

The  formal  ISD  process  is  not  generally  used  when  developing 
training  equipment  for  systems  out  of  acquisition  or  for  common 
training  requirements  across  several  systems.  Interviews  with  ISD 
groups  working  In  these  areas  revealed  several  instances  where  the  ATC 
mandate  for  the  application  of  ISD  to  all  training  development  was 
causing  training  development  groups  to  prepare  formalized  training 
objectives  for  the  training  courses  already  being  taught.  While  this 
exercise  was  useful  In  helping  Instructors  to  tighten  their  Instruc¬ 
tional  regimens.  It  could  have  little  effect  on  making  the  revised 
training  more  Job-relevant,  Since  little  task  data  in  a  formal  sense 
exists  for  most  of  the  systems  out  of  acquisition,  the  preparation  of 
training  objectives  can  be  based  only  on  the  training  course  as  it 
exists,  rather  than  on  any  formal  set  of  Job  performance  requirements. 
Similarly,  the  associated  training  equipment  for  existing  courses  was 
configured  on  the  basis  of  Instructor  preference  and  tradition,  rather 
than  on  the  basis  of  any  formal  analytical  derivation  of  simulation  and 
Instructional  capabilities. 


Job  performance  requirements  and  task  analytical  derivation  of 
traininj^  requirements  is  the  major  key  to  the  effectiveness  of  the  ISO 
process.  With  older  systems  which  have  no  extensive  and  systematic 
task  data  compilations,  it  remains  impractical,  in  most  Instances,  to 
devote  the  time  and  manpower  required  to  amass  such  task  data. 


A  Priori  Simvilator  Selection 


There  is  a  growing  emphasis  within  the  maintenance  training 
development  community  of  the  Air  Force  to  consider  tlie  use  of 
simulators  as  the  primary  training  medium.  Wt»llo  tills  emphasis  forms  a 
very  strong  vote  of  confidence  for  the  maintenance  simulation  movement, 
it  appears  to  reduce  the  already  small  inclination  on  the  part  of  some 
ISO  team  members  to  examine  alternative  means  for  achieving  maintenance 
requirements  and  for  making  cost-effective  training  and  training 
equipment  design  decisions. 

Unfortunately  the  ISO  process  as  it  currently  exists  provides 
little  systematic  guidance  for  the  selection  of  si>eclfic  types  of 
training  equipment.  This  gap  in  the  ISO  procedure  increases  the 
probability  that  training  equipment  selection  will  be  based  on 
preference  rather  than  on  formalized  analyses  aimed  at  maximizing 
cost-effectiveness.  Under  these  conditions  the  uniqueness  of  a 
"simulator"  may  be  easily  justified  for  meeting  training  requirements 
which  could  as  effectively  be  met  by  less  costly  approaclies.  To  the 
credit  of  the  330bth  TiKS,  there  have  been  a  number  of  instances  on 
recent  new  system  ISO  programs  where  simulators  were  not  recommended. 
Ihiwever,  the  emptiasls  remains. 


I nsuf f ic lent  ISO  Team  Training 

Selection  of  Training  Equipment.  AFP  SO- 5 8  provides  little 
guidance  in  the  selection  of  specific  types  of  training  equipment  to 
support  the  achievement  of  specific  training  objectives.  Numerous 
"cons Iderat  ions"  are  suggested,  but  little  formal  structure  Is 
available  to  make  tradeotfs  among  all  of  the  considerations,  leaving 
the  selection  of  trainers  (from  cardboard  mockups  to  full-blown 
simulators)  to  the  SMF's  preference — all  of  this  within  the  broad 
limits  imposed  by  the  procedures  for  using  the  Media  Uecislon  Flow 
Uhart  or  Decision  Table. 

A  great  deal  of  research  over  the  years  has  shown  tliat  for  any  set 
ot  tasks  to  be  trained,  there  are  a  number  of  alternative  combinations 
of  training  media  which  can  be  employed  to  successfully  achieve 
required  learning.  However,  ISO  team  members  are  not  trained  tvi  make 
media  tradeoffs  to  achievo  specific  combinations  ot  learning  capa¬ 
bilities,  ami  to  maximize  the  efficiency  of  a  training  regimen. 
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Training  equipment  permits  student  practice  of  selected  aspects  of 
operational  Jobs,  but  seldom  is  the  provision  for  practice  sufficient 
by  itself.  It  becomes  effective  only  when  integrated  into  a  carefully 
orchestrated  sequence  of  learning  opportunities.  It  is  this 
orchestration  that  is  the  key  to  effective  training.  Selection  of 
specific  training  equipment  must  evolve  from  the  design  of  the  training 
regimen.  Current  training  for  IS!)  team  members  does  not  promote  this 
process. 

Design  of  Training  Equipment  Characteristics.  Once  the  decision 
has  been  made  to  utilize  a  specific  type  of  training  equipment,  a  whole 
new  set  of  training/ learning  implications  becomes  critical.  Tliese 
involve  the  design  of  the  trainer  Itself.  Tliere  are  two  general 
classes  of  decisions: 

1.  What  aspects  of  the  operational  situation  should  be 
simulated,  and  in  what  ways? 

2.  What  instructional  features  or  capabilities,  in  addition 
to  its  simulation  capabilities,  sliould  be  built  into  the 
t  rainer? 

Few  oMSs  prior  to  assignment  to  the  '3306th  T&ES  have  had  the 
opportunity,  especially  in  the  maintenance  training  area,  to 
participate  in  tl\e  design  of  a  major  trainer,  and/or  have  the  formal 
training  in  selectively  employing  the  numerous  state-of-the-art 
approaclies  to  accomplish  particular  training  strategies.  For  example, 
in  the  first  category  (concerning  wiiat  sliould  be  simulated,  and  how) 
there  are  a  large  number  of  considerations  dealing  with  level  of 
fidelity,  such  as: 

1.  Environmental  conditions. 

2.  Stimuli. 

3.  Response  situations. 

4.  Control-display  relationships. 

In  the  second  category  (concerning  what  instructional  features) 
there  are  decisions  concerning  when  and  how  to  employ; 

1.  Enhanced  cueing/feedback. 

2.  Time  distortion  (freeze,  accelerate,  repeat). 


3.  Performance  monitoring. 


4.  Student  station 


S.  Instructor  station. 

Currently,  the  ISD  team  members  are  not  trained  to  deal  with  these 
Issues  in  a  formal  way.  Decisions  Initially  reflect  SMS  experience  and 
preference;  later,  when  the  equipment  is  in  the  procurement  process, 
the  contractor’s  hviman  factors  personnel  may  persuade  the  SMS  to  a^tree 
to  some  enhanced  set  of  equlfwient  character ist Ics.  Despite  the  human 
factors  input,  the  recommended  characteristics  are  seldom  derived  from 
an  Intejtrated  approach  to  meet  the  total  set  of  training  requirements. 


Required  Information  Not  Available 


Traditionally,  systematic  training  and  training  equipment 
develo;imont  efforts,  both  pre-lSD  and  ISD,  suffer  from  difterence  in 
time  phasing  between  system  develoi'ment  and  trainii\g  system 
development.  TtUs  continues  to  be  true  even  with  the  current  ISD 
efforts  on  new  systems  being  conducted  in  a  "fly  before  buv" 
environment,  wliere  tl\e  formal  ISD  effort  is  not  initiated  until  tlie 
prototype  hardware  Is  undergoing  test  and  evaluation.  Three  major 
classes  of  information  currently  appear  to  impede  ISD  progress  earlv  in 
the  analytical  phase: 


1.  Maintenance  and  training  concepts.  Hven  thougit  the 
SON  shovild  Identify  both  maintenance  and  training 
concepts  which  will  be  implemeiUed  on  a  new  svstem, 
coitf  Irmat  ion  and  commitment  to  those  concepts  arc  at 
times,  difficult  for  the  SMSs  ti>  obtain.  As  a 
result  the  ISO  team  must  Intiti.tllv  make 
assvimptions  about  the  vtltimate  maintenance 
orga\»lzat  ion  and  hierarchy  of  tr.tinlivg  experiences 
which  will  prepare  people  tor  t ield  maintenance 
ass ignment  s. 


2,  Trainee  (target)  pop»>lation,  including  levels  aiui 
AKSCs.  There  -tre  often  difficulties  in  obt.tininc, 
using  command  commitments  for  target  populat  ivins 
relevant  to  various  system  maintenance  reqvii  remetU  s  . 
One  consequence  of  this  inability  to  identify  the 
specific  target  pvipulatlon  early  in  the  ISD  prv'cess 
Is  that  assumptions  are  made  aboiit  llie  specific 
AKSOs  auvl  levels  to  N'  .tssigned.  More  important  Iv, 
assumptions  are  made  about  the  recent  weapon  svstt'm 
experience  which  the  target  p<'pulat  ion  will  havi-  as 
thev  enter  training.  The  training  requirements 
themselves  .»re  based  upv'u  the  SMS's  Judgment  v't  lU'w 
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skill  and  knowledge  requirements.  Thus  an  erroneous 
assumption  about  trainee  previous  experience  can 
produce  a  mismatch  between  the  job  performance 
requirements  for  the  new  system  and  the  total  set  of 
skills  and  knowledges  which  the  training  program  is 
designed  to  generate. 

3.  Comprehensive  task  data.  The  task  data  base  for  new 
systems  is  generally  produced  by  the  system 
contractor.  However,  the  system  in  its  prototype 
phase  typically  does  not  have  a  complete  and 
validated  data  base.  Often  subsystems  are  in  a  state 
of  evolution,  making  the  identification  and 
description  of  both  0-  and  I-Level  tasks  difficult. 

There  are  no  ideal  solutions  to  any  of  these  three  major  infor¬ 
mation  area  problems.  Critical  implications  of  them  are:  (1)  the  ISO 
team  as  a  whole  needs  to  very  carefully  coordinate  their  needs  in  order 
to  encourage  and  promote  timely  decision-making  at  the  SPO,  the  using 
command,  and  ATC,  and  (2)  assumptions  need  to  be  made  early  in  the  ISO 
process,  documented,  and  approved/modified  as  Information  becomes 
available.  For  existing  systems,  ISD  teams  need  to  "bite  the  bullet" 
and  quickly  generate  relevant  task  data  prior  to  making  major  training 
and  training  equipment  decisions. 


Incomplete  Analyses 

The  key  to  job-relevant  and  training-effective  ISD  is  comprehen¬ 
sive  task  analysis  information.  It  consists  of  the  specific  skills  and 
knowledges  which  must  be  a  part  of  tiie  job  Incumbent's  repertoire  for 
successful  on-the-job  performance.  To  obtain  skills  and  knowledges,  the 
task  descriptions  (tlie  names  of  specific  tasks  and  steps,  a  description 
of  the  conditions  affecting  performance,  and  a  specification  of  relevant 
performance  standards)  are  analyzed  to  identify  the  specific  beliaviors 
needed  (skills  and  the  application  of  sets  of  knowledges).  The 
behaviors  judged  new  to  the  trainee  .ire  recorded  .is  tr.iinlng  require¬ 
ments  and  .ire  furtlier  an.ilyzed  to  generate  tr. lining  course  and  training 
equipment  implications.  AFP  50-‘i8  describes  tliis  process  in  varying 
detail.  In  actual  use,  lioweve’,  these  procedures  suffer  at  ttiree  m.ijor 
points: 

1.  Task  and  Skill  Analysis.  Figures  3  and  6  in  the 
preceding  subsection  of  tills  report  illustrate  tlie 
typical  contractor  computerized  data  base  (LSA)  and  .i 
completed  FORM  1  which  record  relevant  tasks  not 
found  in  tlie  l.SA  data  base;  (on  some  systems  tliere 

are  no  contractor  produced  data  and  all  task  ^ 
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intorm.it  ion  is  rociirilod  on  tiio  KDRM  1).  Tlie  two 
forms,  lu-r,  oonstituto  I  lu*  typir.il  working  data 

b.iso  for  tlu'  ISl)  team  on  .i  now  systom.  The  problem 
is  tb.it  Ibis  d.it.i  b.iso  consists  of  t.isk  descriptions, 
but  not  dot.iilod  t.isk  an.ilysis.  Required  skills  and 
spec i tic  knowledges  .ire  not  identified.  The 
procedure,  tiowevi’r,  is  to  use  t  be  SMS's  knowledge  of 
simil.ir  systems  .ind  to  do  t  tie  .in.ilysis  "in  tie.id." 

For  .my  step  i>r  .my  t.isk  in  wliicli  .1  new  skill  iir 
knowledge  is  identified,  tbe  SMS  prep. ires  .m  entry  on 
file  Kl'RM  2.  file  R.itionale  ('liecklist  i;.  used  to 
document  e.icb  of  tliese  decisions,  however,  it  does 
not  identify  tbe  specific  skills  .md/or  knowledges 
involved. 

I'be  tlOotti  I'tiKS  procedures  specity  tb.it  beb.ivior.il 
requirements  in  tbe  form  of  skills  .md  knowledges  for 
tr.iining  requirements  bo  identified  on  tbe  FORM  2. 
However,  tbe  FORM  2  often  uses  tbe  s.ime  t.isk 
descriptive  wording  as  tbe  FORM  1  for  each  step  where 
tr.iining  requirements  are  Judged  to  exist,  (tbe  more 
experienced  tbe  SMS  tbe  more  likely  that  tbe 
ciritical  behaviors  will  be  comprehensively 
identified).  If  tbe  step  is  to  remove  I'art  A,  then 
tile  "beb.ivioral  requirement"  on  tbe  FORM  2  will  m.my 
I  lines  be  "remove  I’.irt  .A."  If  the  removal  is 
.issoci.ited  witli  risk  of  equipment  dam.ige,  tbe 
"condition"  column  of  tbe  FORM  1  miglit  c.aution  "avoid 
d.im.ige."  On  tbe  FORM  2,  tbe  Identical  "avoid  damage" 
wording  will  also  be  recorded,  without  specifying  tbe 
p.irt  icul.ir  .ictivities  which  should  be  followed  to 
.ivoid  d.im.ige.  For  SMSs  who  .ire  completely  f.imiliar 
with  .1  p.irt  icul.ir  step  ot  .1  specific  t.isk  (..i 
condition  wliicb  might  be  unlikely  in  .1  new  system), 
tbe  ".ivoid  d.im.ige"  caution  might  be  sufficient  to 
induce  .ippropr  i.ite  recall,  so  that  tbe  specific 
skills  .mil  knowledges  implied  .ire  given  .ippropr i.ite 
cons  ider.it  ion  in  tbe  rem.iining  portivins  of  tbe 
.m.ilysis.  It  is  pvissible,  however,  that  in  complex 
subsystems  some  import. ml  skills  and  knowledges  may 
be  overlooked. 

Tr.iining  F.quipment  Selection.  There  are  sever.il 
problem  .ireas  associated  with  tbe  selection  of 
tr.iining  equipment  to  be  utilized  in  any  of  tbe 
subsvstem  m.i  i  nt  en.ince  tr.iining  programs  for  new 
svstems.  First,  medi.i  selection  is  gener.illy  derived 


from  the  Media  Checklist  and  on  personal  preference, 
and  is  prescribed  to  meet  specific  training 
objectives.  For  training  equipment  this  procedure 
often  results  in  all  identified  training  requirements 
judged  appropriate  for  practice  being  assigned  to  a 
major  simulator,  with  various  audiovisual  media  in 
support.  The  procedure  does  not  guide  decisions 
concerning  the  use  of  other  types  of  practice 
devices,  including  part-task  trainers,  procedures 
trainers,  etc.  Similarly,  the  procedure  does  not 
facilitate  decisions  about  what  set  of  total  practice 
requirements  should  be  incorporated  in  the  simulator; 
two  or  more  simulators  having  limited  application 
might  prove  to  be  more  cost-effective  in  a  given 
training  situation  than  one  major  all-encompassing 
s Imulator. 

Second,  media  selections  are  made  and  documented 
prior  to  the  generation  of  a  POl  which  defines  the 
sequence  and  appropriate  strategies  for  achieving 
specific  training  requirements.  Thus,  often,  tlie  Pc^l 
is  built  around  the  selected  media  rather  than  the 
media  selected  to  most  cost-effectively  support  an 
optimum  instructional  regimen. 

3.  Maintenance  Simulator  Design.  The  operating 
characteristics  of  a  maintenance  simulator  are 
currently  selected,  primarily,  on  the  basis  of  the 
operating  charater ist Ics  of  the  equipment  itself.  To 
paraphrase  one  SMS,  "I'm  not  interested  in  exotic 
features  like  providing  knowledge  of  results.  I  only 
need  the  simulator  to  behave  like  the  airplane,  and  I 
can  teach  with  it."  Consc quent ly ,  most  simulator 
design  recommendations  which  come  from  an  ISO  team  to 
the  SPO  (through  the  TRRRM  and  ATC  review  process) 
essentially  duplicate  actual  equipment  opercjtion. 

This  is  not  necessarily  bad,  but  it  significantly 
reduces  the  probability  that  maxlmiun  training 
usefulness  can  be  d- rived  from  tlie  device. 

While  it  is  easy  to  criticize  the  curre  t  simulator 
design  process,  there  are  not  as  yet  IsD  procedures 
whicli  can  system.it  ical  ly  produce  training  devices 
witti  improved  cost-ef  t oct  iveness. 
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SKCriON’  IV 


rm:  si’o  training  eqi’ifment  acquisition  prockss 


The  System  Program  Of t ice  (SPO)  involvement  with  the  training 
equipment  acquisition  process  is  described  in  tliis  Section  under  three 
main  topics; 


1.  Processes  and  procedures  currently  employed. 

2.  A  general  model  that  sequences  acquisition  decision  sets. 

1,  >1.1  jor  problem  areas. 


Processes  and  Procedures 


The  current  processes  for  procuring  maintenance  training  equipment 
do  not  fit  into  a  single  pattern.  Each  weapon  system  SPO  is  differ¬ 
ently  organized  to  suit  its  function.il  needs.  Tliis  results  in  training 
equipment  m.inagement  .isslgned  to  the  Special  Projects  Office  of  one 
SPO,  to  Logistics  in  another  SPO,  Development  and  iqier.ations  in  a 
third,  etc. 

Broad  categories  of  activities  ot  Tr. lining  Equipment  Acquisition 
Managers,  however,  are  common  to  all  SPOs.  The  activities  for  which 
the  man.iger  is  responsible  include: 

1.  Validation  of  tr.ainlng  device  requirements  as 
presented  by  ATC,  resulting  from  the  ISD 
process. 

2..  Validation  of  tlie  weapon  system  contractor's 
engineering  data. 

3.  Preparation  of  procurement  documentation  which 
tr.insl.ite  ISD-derlved  training  equipment  design 
requirements  into  equipment  functional 
specifications.  The  training  device  acquisition 
goal  is  to  provide  no  more  or  no  less  than  the 
projected  system  requirements. 
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■'» .  o(  v'l'iU  r;U' •  I  iiy;  ri>.*:  1  t'om 

I'l'iit  I'.'ii' I  or  souroo  s»'lo»'ttoii  t  hrintp.li  moii  1 1  or  1  ii}? 

I  tu'  ilt'vo  1  opiiu'nt  proo»'ss  to  .tssiM  i'  tliai  .i 
t  r.i  I  n  1  Oft-o  I  I  oi‘ t  I  v«'  proil»n.-t  In  itoltvoroil  on  tfnio, 

Va  I  iil.it  ion  ol  rr.ifnlnjt  I'l-vii-o  Uoqii  f  r<'iiu‘n(  s 

Tin'  SI’t'  Ao<]ii  I  ;i  1  I  on  M.m.ii'.o  •  '  I  am  i  I  I  .ir  1  .m  t  I  on  witli  I  ho  analvtlo 
a  o  I  1  V  i  I  1 1'S  loaiilnp,  (o  I  !u-  1  SI'  loam  sp«‘i- i  I  t  o  at  I  on  ot  tralnlny,  oi)»\tpmi'nt 
ilosljtn  roipi  ( r<'nn>nt  .'i  pinvl.tos  a  I  I  rm  hauls  lor  I'ovlow  aiul  valtilalton  ol 
tho  roipi  I  ronn'nl  s  In  llp.ht  ol  total  w«'apon  svsiom  proonroinont  i-on- 

tilralnts.  In  ad. ill  Ion,  i  .•.|ii  I  romonl  s  ari>  Inrth.-r  d.'llno.l  hv; 

1.  .M'l'  Inputs  to  tho  Initial  rr.'v'.ram  Mana.’om.-nl  Plan 

that  sniiimail.-o  ro.|nlro.l  pors.ninol  tralnlny.,  an.! 
assnii’  that  I  i  a  I  n  I  no,  .-.'ii  s  1  d.-ra  I  l.nis  liav.'  hoon 
his.'rt.'.l  wlu'i.'v.'i  app  I  I  .'ah  I 

.  Al  t’  Ini'uts  t .'  I  h.'  lrainln,o.  .'.pi  I  pm.-n i  .-..n t  ra.' I  .'r  '  s 

siati'mont  .'l  work. 

t.  Alt'  part  I .' I  pat  I  >>n  In  I  h.'  s.nir.-o  sol.'.'ll.m  pr.'.'.'ss, 

..  I■'l'.'^^n.'n^  hrl.'tlnp,  .'t  t  lu'  SIS'  A.'.pilsl  1  .ni  Mana,o,.'r  t'v 

tho  I  SI'  t  .'am  r.'lal.'.l  t I  h.'  lratnin,o,  .'.piipm.'i'l 
.1.'  r  1  va  I  I  .in  pr.u'.'ss, 

Wh.'ii  I  h.'  I  I  na  1  I /.'.I  trainln}',  .'.pilpuu'nt  r.'.pi  I  r.'m.'iil  s  1  i  .'in  Ih.'  1  SI' 

I  .'.im  a  r.'  I.irmallv  pr.'s.'nt  o.l,  I  h.'  SI’i'  A.'.|n  I  s  1 1  i  on  Manaj',.'i  who  has  K.'pi 

.■iirr.'itl  with  III.'  1  SI'  pr.u-.'ss  is  n.H  .'.m  I  r.mt  o.l  with  snr)'ris.'s.  11.'  .an 
tin'll  r.'a  I  I  s  t  I  .'.I  I  I  V  ass.'ss  t  lu'  ai'pr.'pr  lat  on.'ss  .it  .ir  i  h.'  n.'.'.l  lor  .h'vi.' 
r.'.pi  I  r. '111. 'lit  s  In  ll.o.ht  .i|  I  In'  p.'rsonn.'l  .pia  I  1  I  I  .a  I  I  .ms  n.'.'.'ssaiv  to 
maintain  I  h.'  w.'ap.m  svsl.'iii, 

Vail. lat  l.m  .it  1  nj;  I  n.'.' r  I  no,  I'.ila  an. I  Pr.'paral  l.m  .d  Pr.i.nrom.'iit 
S['o.'  1  t  I  .'a  t  I  .111 

Inp,  I  no.'!' 1  iin  l.'asihtlltv  .it  |ratnin,o,  .'.pi  I  ('iii.'ii  I  imisl  h.'  o.lvon  at  I  .-ii 
'.Ion  in  .'.ms. mail..'  with  .'stimat.'.l  .' I  I I  I  v.'ii.'ss  ,  .'ost,  aii.l  opoiallonal 
t  till.'  .'.mst  ra  1  nt  '. , 

I'tht'i  lasl.'  will. 'll  .'.111  I  r  I  hill  .'  1.'  hotli  tin-  vall.lali.m  ol  d.'vlo.' 
traininp,  .h's.' r  1  pi  I  oiis  aii.l  tho  t  fans  1  a  t  I  .m  ol  I  lioiii  liil.'  pi  .niii  .'iiii'ii  I 
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specializing  in  simulation  research  with  AFHRL  are 
also  available.  Engineering  psychologists  of  the  Human 
Engineering  Laboratory  of  Aerospace  Medical  Division  are 
likewise  available  to  provide  guidance  and  research  support 
directly  to  the  SPO.  Personnel  located  at  the  contractor's 
fabrication  facility  are  assigned  to  the  Air  Force  Plant 
Representative  Office  (AFPRO)  or  to  the  Defense  Contract 
Administrative  Services  (DCAS).  They  may  be  called  upon  by 
the  Acquisition  Manger  to  monitor  progress  directly, 
especially  in  suspected  problem  areas  so  as  to  assure 
equipment  quality  and  to  meet  required  delivery  time 
schedules. 

3.  Air  Force  Documentation.  The  maintenance  training 
equipment  acquisition  team  headed  by  the  SPO 
Acquisition  Manager  has  documentary  resources  to 
assist  in  the  translation  of  training  equipment 
requirements  into  hardware  specifications.  Military 
Standard  490,  "Specification  Practices"  contains  IS 
appendixes  outlining  appropriate  types  of  speci¬ 
fications.  AFHRL-TR-78-28,  see  Bibliography,  pages 
30  and  31  (Hannaman,  Freebie,  &  Miller,  1978),  gives 
the  details  of  a  two-part  specification  process 
normally  employed  in  the  acquisition  of  training 
equipment.  The  first  part  is  the  "Prime  Item 
Development  Specification"  and  the  second  part  Is 
the  "Prime  Item  Product  Fabrication  Specification." 

4.  Contracting  Procedures.  AFHRL-TR-78-28  accurately 
states  that  ttie  SPO  responsibility  for  development 
of  the  specifications  is  usually  accomplished 
ttirough  a  contractor.  The  Prime  Item  Development 
Specification  is  usually  prepared  by  tlie  prime 
weapon  system  contractor  from  the  documented 
training  equipment  requirements  in  conjunction  witli 
ISD  personnel,  and  with  direction,  advice,  and 
approval  of  ttw  SPO  Training  Equi' •..■^nt  Acquisition 
Manager.  Thus,  at  least  for  acqvn..ng  training 
equipment  for  new  systems,  the  requirement  in  AFR 
50-11,  Training,  Management  and  Dtilization  of 
Training  Devices,  that  ISD  personnel  be  included  in 
source  selections  is  superseded  at  an  earlier  point 
in  time  by  t!ie  weapon  system  source  selection. 


Acquisition  Management 

The  weapon  system  contractor's  responsibilities  in  preparation  of 
the  Prime  Item  Development  Specification  for  maintenance  training 
equipment  are  usually  established  at  system  source  selection.  Tills 
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decision  has  customarily  been  made  because  the  contractor  lias  ready 
access  to  the  necessary  engineering  data  and  expertise. 

In  most  SPO  maintenance  training  equipment  acquisiton  programs,  a 
departure  from  the  process  prescribed  by  directives  often  occurs  at  the 
point  when  the  Prime  Item  Development  Specification  for  the  weapon 
system  is  approved.  Most  often,  as  previously  described,  the  prime 
weapon  system  contractor  is  awarded  training  equipment  delivery 
responsibi 1 ity . 

The  problem  causing  the  change  in  the  acquisition  process,  as 
stated  by  regulations,  occurs  because  the  prime  contractor  usually 
subcontracts  the  training  equipment  development  and  production  effort. 
Armed  Services  procurement  policy  discourages  technical  inputs  to  the 
subcontracted  process  by  Air  Force  monitors.  Direction  of  this  type 
may  give  the  prime  contractor  reasons  to  demand  additional  funding  or  a 
modification  of  critical  delivery  schedules.  Additional  conflicts 
occur  since  regulations  require  the  acquisition  manager  and  team  to 
maintain  visibility  and  control  throughout  the  development  and 
production  programs. 

Usually,  the  Intercontinental  Ballistic  Missile  (ICBM)  SPO  of 
Space  and  Missile  Systems  Organization(SAMSO)>^n>'>tracts  with  the  weapon 
system  contractor  for  training  equipment  and  simulators.  This 
contractor  does  not  subcontract.  Therefore,  the  ICBM  SPO  does  not 
encounter  the  problem  of  maintaining  program  control  and  visibility. 

In  other  instances  when  the  ICBM  SPO  contracts  directly  for 
training  equipment  from  other  than  tlie  weapon  system  contractor,  the 
management  problems  noted  above  are  lessened  by  contracting  for  .in 
engineering  source  data  package  called  System  Requirements  Analvsis, 
(SRAl  from  the  prime  contractor. 

The  SRA  package  supplied  by  the  prime  weapon  system  contractor  is 
defined  In  SAMSO-STD-77-6  as  "A  sequential  and  iterative  engineering 
process  designed  to  establish  the  functional  requirements  for  each 
element  of  a  weapon  system.  The  process  provides  a  logical  sequence 
and  .«  clear  record  of  the  development  of  system  requirements  to  m.inage 
the  system  engineering  effort  throughout  all  phases  of  system 
acquisition."  Tliis  analysis  systematically  establishes  requirements 
for  equipment,  personnel,  procedures,  and  facilities,  Pigure  I'i 
Indicates  SRA  inputs  to  i'>ersonnel  and  training  development  processes. 

Engineering  data  and  support  from  weapon  svstom  contractors  for 
aircraft  and  electronic  system  SPOs  are  often  variable  in  quality. 
Frequently  they  are  not  available  in  time  to  meet  training  definition 
requirements . 
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The  apparent  orvtical  differences  between  the  SRA  approach  and 
tliat  followed  for  aircraft  weapon  system  contractors  are: 

1.  The  SRA  data  package  has  its  own  detailed  and 
explicit  specification,  rather  than  being  an 
associated  output  from  the  weapon  system  prime 
item  specification. 

2.  Data  production  milestones  and  quality  are  closely 
monitored  during  production. 

3.  Data  outputs  are  phiised  with  weapon  system 
development  so  titat  updated  and  expanded  packages 
are  produced  as  the  system(s)  are  designed.  In 
this  way,  preliminary  data  are  released  early, 
with  more  detail  evolving  as  development 
progresses. 

Other  maintenance  training  equipment  management  differences  among 
SPOs  are  caused  by  varying  degrees  of  participation  by  engineering 
specialists  in  the  decision-making  process  that  supports  the  SPO 
Acquisition  Manager.  Engineer  availability,  personnel  expertise  level, 
and  the  SPO's  willingness  to  consult  the  engineers  are  additional 
factors  which  affect  decisions. 


A  Training  Equipment  Acquisition  Process  Model 

The  managerial  decisions  involved  in  the  SPO  training  equipment 
acquisition  process  are  identified  and  sequenced  in  the  model  that 
follows.  In  an  attempt  to  accommodate  the  wide  range  of  existing 
contractual,  organi  7,at  ional ,  and  managerial  differences  noted  above, 
the  model  is  necessarily  very  general.  It  is,  however,  applicable  to 
most  acquisition  procedures.  The  decisions  required  in  this  process 
are  indicated  irrespective  of  the  agency  who  may  make  them,  whether  it 
be  the  SPO  Manager,  a  contractor.  Air  Force  engineers,  or  an  acqui¬ 
sition  team. 

The  training  equipment  acquisition  process  model  begins  with 
training  equipment  design  documents  produced  by  the  ISD  team.  Appli¬ 
cation  of  the  recommended  ISD  procedures  and  the  employment  of  the 
training  equipment  design  process  model  proposed  earlier  structure  the 
SPO  inputs.  Even  efficiently  managed  acquisition  effort  may  be  wasted 
if  the  procured  equipment  has  been  based  on  an  Incomplete  design 
process . 


Tho  nm  lor  parts  of  the  Sf'H  ao<\u(sltlon  dorlsloa  |iroi-oss  iarliulo 
tho  dovo  loiwu'ut  of  ai)  offootlvo  prooiirt'inoiU  spoi- i  1  loat  loa,  t  lu- 
maaaKonuMU  of  tho  dovelopmoiU  aiuf  fabrloaflon  proross,  aiui  tla- 
ovaluatlon  of  the  delivered  trainer. 


A  I’roeedora  I  /  Dee  1  s  Ion  Se^nenee 

Kfunre  Ih  deplets  the  pri>eednra  I /dee  Is  f  on  proeess  and  snmmar  I /.es 
tlie  supporting;  inputs  neeessary  lor  aeeomp  1  1  slunent .  The  nia  |or  steps  ol 
the  si'ipieitt’i'  Involve; 

1.  Valldite  the  training  eiiulpmeni  lunetlon  and  design 
eharae t er 1 s t  ies  doeumented  as  a  result  of  t  lu‘  ISD 
proeess . 

2.  Dett'rmlne  the  feasibility  ol  the  vallilatml  »'c|ul|Mnent 
rei|u  1  rement  s  In  t»'rms  ol  available  immetarv  resoure<> 
estimates,  delivery  time  reijul  rement  s  ,  and 

eng  I  lu'i'f  I  ng  st  a  t  e-ot  -  t  he-art  . 

t.  I’la'sent  just  1 1  feat  Ion  rationale  t>>  the  Sl'(>  I'roi’.ram 
nireetor  toi-  approval  of  neeil  and  alloeations. 

•t.  I’repar<>  Statement  ol  Work  (StfWl  and  Kecpiest  lor 

Proposal  (UKl’)  doeument  at  Ion  detailing  I  h(>  management 
approaeh  appl  U-ahle  to  eontraetor  aetivltles. 


ii. 


Seli'i't  eoniraetual  souree  bv  eomparat  1  ve  I  v  assessing 
proposals  on  the  basis  ol  doeument  eil  tei'lmli'il 
app''oael>,  understanding  ot  r»'i)u  I  rianent  s  ,  Innovations 
towarvl  satisfying  goals,  timely  prodiu-t  vh'llvery, 
t'xper  1  enei' ,  laellitles,  personnel  resourees,  .ind 

i'OSt  . 

Reevaluate  and  tinall/.e  del  ills  of  t  lu'  proeurement 
spi>e  1  I  1 1  a t  1  on  to  assure  lameurrenee  witli  everv 
speeltie  reipi  1  rement  ,  emphasl/.ing  to  t  lie  i-ontraetor 
that  the  rigorous  lest,  aeeeplanee,  aiul  v'lu>ekoiit 
proeediires  (-onta'iied  in  the  spee  I  I  I  ea  t  I  on  will  be 
St  r  WM  1  V  enl  on  ed. 


/.  Monitor,  within  eont  rae  t  iia  1  1  v  legal  hounds,  the 
devi'lofwu'nt  al  and  produet  Ion  proeess  to  assure 
eniilpment  and  timeliness  i)l  ei|ulpmenl  delivery. 
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.  Suporvlso  aiul  part  (o  lp.it  o  (ii  tlu'  spoo  1 1  loti  tost, 
aocoptaiK'o,  aiiil  oliookout  ;io  t  1  v  ( t  los .  t^inirJ  i  n.it  i' 
iislnn  oomm.inJ  aiul  oxport  onginoi'ri  up,  support  to 
assure  that  the  m.itutou.iuoo  tr.iiniup,  oipiipmoiil  nu't'ts 
0  011 1  rao  toil  roipi  i  romoiit  s . 


Major  I’robloms  Areas 


As  .1  result  ot  the  review  ot  doeumeut  at  iou,  the  ohserv.it  i  ouis 
SPO  procedures  and  .i  study  ot  the  tr.ilniiij;  ei]ulpnieiit  .lonu  i  s  1 1  i  on 
vieoisions  retpilred,  three  prtihlem  areas  liave  ht'en  identil  ied: 

1.  V.irlahle  M.in.ipement  Pr.iotioes. 

2.  hack  lit'  Proeedur.it  laiidanee. 

3.  h.ite  Acquisition. 


Variable  Manat’ement  Practices 

The  variability  ot  cont  r.io  t  ua  1  organ  i /.at  ion. 1 1  .uul  in.inagi'ri.i  I 
ch.iracterlstlcs  oV  the  acquisition  process  oiten  ret  loot  .i  rclativelv 
low  priority  that  malnten.ince  training  equipment  h.is  in  comi'.irison  to 
other  are.'is  of  tlie  system  development  process.  L.ick  ot  consistent 
organ  I /, at  ion  among  SPDs  h.is  resulted  in  v.irying  degrees  oi  program 
support  to  m.i  i  ntenance  tr. lining  equipment  ni.in.ip,ement  and,  subsequent  1  \  , 
to  variable  training  equipment  qualitv. 

The  single  point  procurement  m.in.igenu’nt  approach  emploved  liv  S.VMSO 
has  recently  been  implemented  for  .leronaut  ical  svstems.  SlMSl'ii  h.is 
been  design.ited  to  .issume  tills  responsibility.  This  org.ini  .-at  ion  h.is 
extensive  experience  in  ra.in.iging  the  .icquisition  ot  flight  crew 
simulators.  This  experience  should  effecllvelv  gener.ili.re  to  m.iin- 
tenance  training  slmul.itors. 

The  major  .ulvant.iges  of  h.ivlng  .ill  maintenance  tr. lining 
simulators  acquired  through  SIMSPO  include: 

1.  .St  andard  I  z.it  Ion  of  cont  r.ict  i  ng  I'rocedures  that 
establish  a  set  of  working  arrangements  with 
contracting  officers  and  between  team  members  ot 
S IMSPd. 

2.  i't  l  I  i /at  ion  ot  Its  own  c.idre  ot  m.i  i  nt  I'li.ince 
slmul.itor  experts  to  de.il  with  decisions  requiring 
luim.in  f.ictors  .itui  eng  I  ne»'r  i  ng  exi'erlise. 
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3.  Coordinated  channels  with  Air  Force  laboratory  and 
engineering  experts  who,  at  times,  may  be  needed  to 
establish  specifications  and/or  evaluation  designs. 


Lack  of  Procedural  Guidance 


A  primary  directive  in  the  acquisition  of  maintenance  training 
equipment  Is  MlL-T-81821,  Military  Specification:  Trainers,  Main¬ 
tenance,  Equipment  and  Services,  General  Specifications  For .  This 
specification,  in  the  opinion  of  the  authors,  appears  to  place  require¬ 
ments  for  realism  and  functional  fidelity  which  can  adversely  affect 
the  ultimate  cost-effectiveness  of  maintenance  trainers,  especially 
simulators.  The  directive  also  necessitates  extensive  justification 
effort  if  the  requirements  are  deviated  from  to  achieve  enhanced 
instructional  value. 

Under  conditions  where  maintenance  simulators  are  being  procured 
through  the  weapon  system  SPO,  this  directive  is  particularly  cumber¬ 
some.  Many  Training  Kquipment  Acquisition  Managers  are  new  to  their 
Jobs  and  are  relatively  unfamiliar  with  maintenance  training  and  with 
equipment  procurement.  For  them,  MIL-T-81821  can  he  especially 
misleading  in  guiding  the  preparation  of  procurement  specifications. 

With  the  shift  of  acquisition  responsibility  for  maintenance 
simulators  to  SIMSPO  the  detrimental  effects  of  MlL-T-81821  will  be 
reduced.  Highly  experienced  specification  writers  will  be  able  to 
selectively  apply  the  requirements  to  acliieve  the  training  effective¬ 
ness  identified  and  recommended  by  the  ISD  analysts.  On-going  studies 
of  maintenance  training  equipment  design  and  acquisition  stiould  produce 
ttu'  principals  and  procedures  to  ix'rmlt  future  updating  of  Mn--T-8182l. 


Late  Acquisition 

The  most  Important  problem  area  in  tlie  acquisition  process  Is  that 
the  definition  of  training  equipment  requirements  and  tlie  subsequent 
Issuance  of  a  procurement  specification  are  not  accompl istied  in  time  to 
permit  trainer  delivery  in  support  of  initial  oper.itlonal  training. 

There  are  several  related  factors  involved.  I.att*  receipt  and  lack  i>f 
completeness  of  engineering  and  task  data  provided  to  the  ISU  teams 
prolongs  tiu’  equipment  design  (x'rlod. 

1.  A  staff  study  conducted  and  reported  by  Aero¬ 
nautical  Systems  I)lvlsl<>n  recommends  centralized 
procurement  «>f  maintenance  simulators  for  aircraft 
systems  In  ASD's  Simulator  SPG  (SIMSPO). 

1 
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A  paraj^rapti  of  tho  study  statos:  "...tlu'  ma  pir 
funding  problem  was  tho  need  for  using  training 
equipment  funds  to  cover  overruns  in  the  air 
vehicle  system."  (Maintenance  Training  Simulator 
Procurement ,  page  8).  Karller  training  equipment 
requirement  definition  could  have  provided 
contractual  utilization  of  budgeted  funds  before 
they  were  used  for  other  purposes. 

2.  The  late  definition  of  requirements  sometimes 
results  in  a  contract  which  combines  research/- 
development  and  fabrication  phases.  These 
procurement  iiractices  occur  in  an  effort  to  meet 
required  equipment  delivery  dates  by  attempting  to 
make  up  time  lost  in  deflniitg  requirements.  This 
type  of  contract  often  results  in  funding 
difficulty  due  to  an  escalation  of  device 
complexity  and  price. 

In  addition,  there  are  times  when  insufficient 
manpower  avallahllity  among  engineering  advisors 
whose  time  is  shared  among  acquisition  programs  of 
several  systems  delays  completion  of  the  spec¬ 
ification.  Finally,  Itigh  turnover  rates  amo\^g  SPO 
Acquisition  Managers  during  the  procurement  process 
dtie  to  military  transfers  furtlior  slows  tlie 
procurement  process  and  detracts  from  the  avail¬ 
ability  and  cost-effectiveness  of  the  final 
training  system. 
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